2024-04-25T11:31:30.695 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T11:31:30.695 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332 --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/suites/pjd}} --name leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose -- /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/orig.config.yaml 2024-04-25T11:31:30.742 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:31:30.828 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332 branch: wip-lusov-quiescer description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/suites/pjd}} email: null first_in_suite: false job_id: '7673332' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 fuse set user groups: true rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: df40303764f976ac2e973c1ef2b40d1ebbda473b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 2866 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: df40303764f976ac2e973c1ef2b40d1ebbda473b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJNl3N2R7LyI0uZGFSlGW1HbkwyyfKdvKA0bEJ5k3GWtbtg5eNAT09UYtGbAozRjlBR7SLesr+YXP/WxnAt3IM= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJktnmc6KegBeiC88dl+vdB9DEb/41t0qgszN4lGnfkUjzaK+GY6tk2mzVIgxPl2AmEqcIYAtrV7imk9k1IzJ0k= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - quiescer: null - workunit: clients: all: - suites/pjd.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_11:13:36 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T11:31:30.828 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa; will attempt to use it 2024-04-25T11:31:30.829 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks 2024-04-25T11:31:30.829 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T11:31:30.831 INFO:teuthology.task.internal:Checking packages... 2024-04-25T11:31:30.854 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T11:31:30.854 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T11:31:30.855 INFO:teuthology.packaging:ref: None 2024-04-25T11:31:30.855 INFO:teuthology.packaging:tag: None 2024-04-25T11:31:30.855 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T11:31:30.855 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:31:30.855 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T11:31:31.111 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T11:31:31.112 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T11:31:31.119 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T11:31:31.119 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T11:31:31.175 INFO:teuthology.task.internal:Saving configuration 2024-04-25T11:31:31.192 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T11:31:31.198 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:31:31.220 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:26:32.830490', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJNl3N2R7LyI0uZGFSlGW1HbkwyyfKdvKA0bEJ5k3GWtbtg5eNAT09UYtGbAozRjlBR7SLesr+YXP/WxnAt3IM='} 2024-04-25T11:31:31.237 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:26:32.828375', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJktnmc6KegBeiC88dl+vdB9DEb/41t0qgszN4lGnfkUjzaK+GY6tk2mzVIgxPl2AmEqcIYAtrV7imk9k1IzJ0k='} 2024-04-25T11:31:31.237 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T11:31:31.243 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-25T11:31:31.243 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-25T11:31:31.243 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T11:31:31.309 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4479bcc0d0>, signals=[15]) 2024-04-25T11:31:31.310 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T11:31:31.316 INFO:teuthology.task.internal:Opening connections... 2024-04-25T11:31:31.316 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:31:31.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:31:31.398 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:31:31.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:31:31.476 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T11:31:31.484 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-04-25T11:31:31.503 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-04-25T11:31:31.503 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-04-25T11:31:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-04-25T11:31:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-04-25T11:31:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-04-25T11:31:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:31:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-04-25T11:31:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:31:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:31:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-04-25T11:31:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:31:31.562 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:31:31.562 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:31:31.562 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:31:31.562 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:31:31.562 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:31:31.562 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-04-25T11:31:31.584 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-04-25T11:31:31.603 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-04-25T11:31:31.603 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-04-25T11:31:31.661 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-04-25T11:31:31.661 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-04-25T11:31:31.661 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-04-25T11:31:31.661 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:31:31.661 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-04-25T11:31:31.662 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:31:31.662 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:31:31.662 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-04-25T11:31:31.662 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:31:31.662 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:31:31.662 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:31:31.662 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:31:31.662 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:31:31.662 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:31:31.662 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-04-25T11:31:31.692 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T11:31:31.702 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T11:31:31.726 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T11:31:31.726 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:31:31.729 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:31:31.746 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T11:31:31.754 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T11:31:31.755 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:31:31.787 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:31:31.809 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T11:31:31.866 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T11:31:31.880 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T11:31:31.880 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T11:31:31.881 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:31:31.881 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:31:31.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:31:31.898 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-04-25T11:31:31.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:31:31.899 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-04-25T11:31:31.956 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-437.el9.x86_64 2024-04-25T11:31:31.956 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-437.el9.x86_64 2024-04-25T11:31:31.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2024-04-25T11:31:31.957 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-437.el9.x86_64 2024-04-25T11:31:31.958 INFO:teuthology.task.kernel:Running kernel on smithi062: 5.14.0-437.el9.x86_64 2024-04-25T11:31:31.958 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2024-04-25T11:31:34.319 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T11:31:36.270 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T11:31:36.460 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.0 MB 00:02 2024-04-25T11:31:39.327 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T11:31:40.953 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.2 MB 00:00 2024-04-25T11:31:44.119 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.2 MB 00:01 2024-04-25T11:31:45.562 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-04-25T11:31:47.013 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T11:31:48.120 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 2.7 kB/s | 16 kB 00:05 2024-04-25T11:31:49.441 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T11:31:52.322 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T11:31:54.319 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T11:31:54.319 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T11:31:54.414 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-25T11:31:54.415 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-04-25T11:31:54.415 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-25T11:31:54.536 DEBUG:teuthology.orchestra.run.smithi062:> echo no | sudo yum reinstall kernel || true 2024-04-25T11:31:54.630 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T11:31:55.095 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:31:52 AM UTC. 2024-04-25T11:31:55.227 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-25T11:31:55.242 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-25T11:31:55.242 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-04-25T11:31:55.242 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-25T11:31:55.242 INFO:teuthology.orchestra.run.smithi062.stdout:Reinstalling: 2024-04-25T11:31:55.243 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:31:55.243 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:31:55.243 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-25T11:31:55.243 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-25T11:31:55.243 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:31:55.243 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 255 k 2024-04-25T11:31:55.243 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 0 2024-04-25T11:31:55.243 INFO:teuthology.orchestra.run.smithi062.stderr:Operation aborted. 2024-04-25T11:31:55.301 INFO:teuthology.orchestra.run.smithi062.stdout:Is this ok [y/N]: 2024-04-25T11:31:55.301 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum reinstall -y kernel || true 2024-04-25T11:31:55.818 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:31:52 AM UTC. 2024-04-25T11:31:55.939 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-25T11:31:55.940 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-25T11:31:55.940 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-04-25T11:31:55.940 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-25T11:31:55.940 INFO:teuthology.orchestra.run.smithi062.stdout:Reinstalling: 2024-04-25T11:31:55.940 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:31:55.940 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:31:55.940 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-25T11:31:55.940 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-25T11:31:55.940 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:31:55.941 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 255 k 2024-04-25T11:31:55.941 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 0 2024-04-25T11:31:55.941 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-04-25T11:31:56.190 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-25T11:31:56.191 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:31:56.191 INFO:teuthology.orchestra.run.smithi062.stdout:Total 1.0 MB/s | 255 kB 00:00 2024-04-25T11:31:56.211 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-04-25T11:31:56.400 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-04-25T11:31:56.400 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-04-25T11:31:56.405 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-04-25T11:31:56.405 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-04-25T11:31:56.560 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-04-25T11:31:56.611 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T11:31:56.611 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T11:31:56.641 INFO:teuthology.orchestra.run.smithi062.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:31:56.641 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:31:56.716 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-25T11:31:56.718 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-25T11:31:56.718 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-25T11:31:56.816 DEBUG:teuthology.orchestra.run.smithi040:> echo no | sudo yum reinstall kernel || true 2024-04-25T11:31:57.299 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:31:57.299 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:31:57.342 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:31:54 AM UTC. 2024-04-25T11:31:57.466 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-25T11:31:57.479 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T11:31:57.479 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-25T11:31:57.479 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T11:31:57.479 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-25T11:31:57.479 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:31:57.479 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:31:57.480 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-25T11:31:57.480 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T11:31:57.480 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:31:57.480 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-25T11:31:57.480 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-25T11:31:57.480 INFO:teuthology.orchestra.run.smithi040.stderr:Operation aborted. 2024-04-25T11:31:57.551 INFO:teuthology.orchestra.run.smithi040.stdout:Is this ok [y/N]: 2024-04-25T11:31:57.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum reinstall -y kernel || true 2024-04-25T11:31:58.053 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:31:58.053 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:31:58.053 INFO:teuthology.orchestra.run.smithi062.stdout:Reinstalled: 2024-04-25T11:31:58.053 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T11:31:58.053 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:31:58.053 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-25T11:31:58.064 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 11:31:54 AM UTC. 2024-04-25T11:31:58.185 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-25T11:31:58.186 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T11:31:58.186 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-25T11:31:58.186 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T11:31:58.186 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-25T11:31:58.186 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:31:58.186 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:31:58.186 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-25T11:31:58.187 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-25T11:31:58.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:31:58.187 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-25T11:31:58.187 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-25T11:31:58.187 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-25T11:31:58.241 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T11:31:58.290 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T11:31:58.290 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T11:31:58.290 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T11:31:58.290 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T11:31:58.290 DEBUG:teuthology.orchestra.run.smithi062:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T11:31:58.325 INFO:teuthology.orchestra.run.smithi062.stdout:ttyS1 2024-04-25T11:31:58.349 DEBUG:teuthology.parallel:result is None 2024-04-25T11:31:58.532 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.0 MB/s | 255 kB 00:00 2024-04-25T11:31:58.532 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:31:58.532 INFO:teuthology.orchestra.run.smithi040.stdout:Total 737 kB/s | 255 kB 00:00 2024-04-25T11:31:58.556 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-25T11:31:58.761 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-25T11:31:58.761 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-25T11:31:58.766 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-25T11:31:58.766 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-25T11:31:58.983 INFO:teuthology.orchestra.run.smithi040.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:31:58.984 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:31:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:31:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:32:00.338 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:32:00.338 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:32:00.338 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalled: 2024-04-25T11:32:00.338 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T11:32:00.338 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:32:00.338 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-25T11:32:00.520 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T11:32:00.567 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T11:32:00.568 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T11:32:00.568 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T11:32:00.568 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T11:32:00.568 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T11:32:00.601 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2024-04-25T11:32:00.620 DEBUG:teuthology.parallel:result is None 2024-04-25T11:32:00.620 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T11:32:00.629 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T11:32:00.629 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:32:00.662 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:32:00.681 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T11:32:00.689 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T11:32:00.695 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T11:32:00.695 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:32:00.717 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:32:00.748 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T11:32:00.755 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T11:32:00.755 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:32:00.778 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:32:00.809 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:32:00.818 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:32:00.836 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:32:00.847 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:32:00.849 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T11:32:00.857 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T11:32:00.857 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:32:00.862 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:32:00.889 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T11:32:00.897 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T11:32:00.898 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:32:00.926 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:32:00.945 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:32:01.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:32:01.067 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:32:01.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:32:01.189 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:32:01.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:32:01.253 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:32:01.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:32:01.359 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:32:01.404 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:32:01.445 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:32:01.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:32:01.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-04-25T11:32:01.515 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-04-25T11:32:01.547 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:32:01.593 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:32:02.004 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T11:32:02.013 INFO:teuthology.task.internal:Starting timer... 2024-04-25T11:32:02.013 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T11:32:02.034 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T11:32:02.043 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T11:32:02.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-04-25T11:32:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-04-25T11:32:02.118 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-04-25T11:32:02.174 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-04-25T11:32:02.176 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T11:32:02.176 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-04-25T11:32:02.199 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-04-25T11:32:02.199 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-04-25T11:32:02.217 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-04-25T11:32:02.217 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi062.front.sepia.ceph.com': 'permissive'} 2024-04-25T11:32:02.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:32:02.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:32:02.266 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:32:02.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:32:02.292 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T11:32:02.292 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:32:02.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:32:02.386 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T11:32:02.395 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T11:32:02.401 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T11:32:02.523 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T11:32:02.539 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T11:32:02.540 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi062.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T11:35:38.352 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com')] 2024-04-25T11:35:38.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-25T11:35:38.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:35:38.433 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-25T11:35:38.509 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-25T11:35:38.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T11:35:38.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:35:38.587 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-25T11:35:38.666 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T11:35:38.667 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T11:35:38.678 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T11:35:38.678 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:35:38.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:35:38.681 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:35:38.681 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:35:38.710 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:35:38.721 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:35:38.738 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:35:38.745 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-04-25T11:35:38.752 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:35:38.756 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-04-25T11:35:38.769 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:35:38.776 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-04-25T11:35:38.783 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:35:38.787 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-04-25T11:35:38.801 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:35:38.814 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:35:38.840 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:35:38.843 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:35:38.843 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-25T11:35:38.843 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:35:38.843 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:35:38.843 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:35:38.843 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:35:38.867 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:35:38.870 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:35:38.871 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-25T11:35:38.871 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:35:38.871 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:35:38.871 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:35:38.871 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:35:38.872 INFO:teuthology.run_tasks:Running task install... 2024-04-25T11:35:38.880 DEBUG:teuthology.task.install:project ceph 2024-04-25T11:35:38.880 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T11:35:38.880 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T11:35:38.881 INFO:teuthology.task.install:Using flavor: default 2024-04-25T11:35:38.886 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T11:35:38.886 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T11:35:38.887 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T11:35:38.887 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:35:38.890 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T11:35:38.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:35:39.043 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T11:35:39.043 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T11:35:39.113 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T11:35:39.113 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T11:35:39.155 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:35:39.155 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:35:39.155 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:35:39.181 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:35:39.182 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:35:39.225 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:35:39.225 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:35:39.225 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:35:39.248 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:35:39.251 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:35:39.251 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:35:39.316 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:35:39.323 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:35:39.387 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-04-25T11:35:39.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-04-25T11:35:39.390 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:35:39.454 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-04-25T11:35:39.455 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-04-25T11:35:39.622 INFO:teuthology.orchestra.run.smithi062.stdout:48 files removed 2024-04-25T11:35:39.649 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:35:39.689 INFO:teuthology.orchestra.run.smithi040.stdout:48 files removed 2024-04-25T11:35:39.717 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:35:40.393 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 207 kB/s | 83 kB 00:00 2024-04-25T11:35:40.654 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 114 kB/s | 83 kB 00:00 2024-04-25T11:35:40.762 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-25T11:35:41.046 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-25T11:35:41.144 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-04-25T11:35:41.413 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-25T11:35:41.727 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T11:35:42.688 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.0 MB 00:01 2024-04-25T11:35:43.603 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-25T11:35:45.448 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T11:35:47.854 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T11:35:49.907 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.2 MB 00:00 2024-04-25T11:35:50.813 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 9.2 kB/s | 16 kB 00:01 2024-04-25T11:35:51.574 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-25T11:35:52.868 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T11:35:53.623 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 21 MB 00:02 2024-04-25T11:35:58.076 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T11:35:58.856 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-25T11:36:00.311 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:36:00.312 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:36:00.323 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:36:00.324 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:36:00.325 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:36:00.325 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:36:00.327 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:36:00.329 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:36:00.330 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:36:00.394 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-25T11:36:00.403 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-25T11:36:00.403 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-04-25T11:36:00.403 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-25T11:36:00.403 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-04-25T11:36:00.403 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:36:00.403 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T11:36:00.403 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:36:00.403 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T11:36:00.403 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:36:00.404 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T11:36:00.405 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T11:36:00.406 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:36:00.407 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:36:00.408 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T11:36:00.409 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:36:00.410 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:36:00.411 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:36:00.412 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:36:00.413 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:36:00.414 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:36:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-04-25T11:36:00.414 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:36:00.414 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:36:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-25T11:36:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-25T11:36:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:Install 148 Packages 2024-04-25T11:36:00.414 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 4 Packages 2024-04-25T11:36:00.414 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:36:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 222 M 2024-04-25T11:36:00.415 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-04-25T11:36:01.084 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:36:01.084 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:36:01.096 INFO:teuthology.orchestra.run.smithi040.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:36:01.096 INFO:teuthology.orchestra.run.smithi040.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:36:01.097 INFO:teuthology.orchestra.run.smithi040.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:36:01.098 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:36:01.100 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:36:01.102 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:36:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:36:01.166 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T11:36:01.176 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T11:36:01.177 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T11:36:01.178 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:36:01.179 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:36:01.180 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:36:01.181 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:36:01.182 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:36:01.183 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:36:01.184 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:36:01.185 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:36:01.186 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:36:01.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:36:01.188 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-25T11:36:01.188 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-25T11:36:01.188 INFO:teuthology.orchestra.run.smithi040.stdout:Install 148 Packages 2024-04-25T11:36:01.188 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 4 Packages 2024-04-25T11:36:01.188 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:36:01.188 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 222 M 2024-04-25T11:36:01.188 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-25T11:36:01.752 INFO:teuthology.orchestra.run.smithi062.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-25T11:36:02.250 INFO:teuthology.orchestra.run.smithi040.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-25T11:36:02.462 INFO:teuthology.orchestra.run.smithi062.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 890 kB 00:00 2024-04-25T11:36:02.578 INFO:teuthology.orchestra.run.smithi062.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.2 MB/s | 145 kB 00:00 2024-04-25T11:36:02.783 INFO:teuthology.orchestra.run.smithi040.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 890 kB 00:00 2024-04-25T11:36:02.875 INFO:teuthology.orchestra.run.smithi040.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.5 MB/s | 145 kB 00:00 2024-04-25T11:36:03.228 INFO:teuthology.orchestra.run.smithi062.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 2.4 MB 00:00 2024-04-25T11:36:03.312 INFO:teuthology.orchestra.run.smithi062.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.7 MB/s | 5.4 MB 00:01 2024-04-25T11:36:03.545 INFO:teuthology.orchestra.run.smithi062.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 4.7 MB/s | 1.5 MB 00:00 2024-04-25T11:36:03.617 INFO:teuthology.orchestra.run.smithi040.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.2 MB/s | 2.4 MB 00:00 2024-04-25T11:36:04.092 INFO:teuthology.orchestra.run.smithi040.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-25T11:36:04.421 INFO:teuthology.orchestra.run.smithi062.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 6.8 MB/s | 21 MB 00:03 2024-04-25T11:36:04.518 INFO:teuthology.orchestra.run.smithi040.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-25T11:36:04.579 INFO:teuthology.orchestra.run.smithi062.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.7 MB/s | 4.7 MB 00:01 2024-04-25T11:36:05.104 INFO:teuthology.orchestra.run.smithi062.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 48 kB/s | 25 kB 00:00 2024-04-25T11:36:05.584 INFO:teuthology.orchestra.run.smithi040.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.1 MB/s | 4.7 MB 00:01 2024-04-25T11:36:05.988 INFO:teuthology.orchestra.run.smithi062.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 6.8 MB/s | 11 MB 00:01 2024-04-25T11:36:06.157 INFO:teuthology.orchestra.run.smithi062.stdout:(11/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 6.4 MB/s | 17 MB 00:02 2024-04-25T11:36:06.180 INFO:teuthology.orchestra.run.smithi062.stdout:(12/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.1 MB/s | 225 kB 00:00 2024-04-25T11:36:06.263 INFO:teuthology.orchestra.run.smithi062.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 250 kB/s | 27 kB 00:00 2024-04-25T11:36:06.289 INFO:teuthology.orchestra.run.smithi062.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 292 kB/s | 32 kB 00:00 2024-04-25T11:36:06.381 INFO:teuthology.orchestra.run.smithi062.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 163 kB 00:00 2024-04-25T11:36:06.456 INFO:teuthology.orchestra.run.smithi062.stdout:(16/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.8 MB/s | 747 kB 00:00 2024-04-25T11:36:06.481 INFO:teuthology.orchestra.run.smithi062.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.2 MB/s | 126 kB 00:00 2024-04-25T11:36:06.566 INFO:teuthology.orchestra.run.smithi062.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 4.5 MB/s | 500 kB 00:00 2024-04-25T11:36:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 361 kB/s | 45 kB 00:00 2024-04-25T11:36:06.799 INFO:teuthology.orchestra.run.smithi062.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.2 MB/s | 135 kB 00:00 2024-04-25T11:36:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.5 MB/s | 161 kB 00:00 2024-04-25T11:36:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 320 kB 00:00 2024-04-25T11:36:07.150 INFO:teuthology.orchestra.run.smithi062.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 298 kB 00:00 2024-04-25T11:36:07.267 INFO:teuthology.orchestra.run.smithi062.stdout:(24/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 6.8 MB/s | 5.4 MB 00:00 2024-04-25T11:36:07.292 INFO:teuthology.orchestra.run.smithi062.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 700 kB/s | 99 kB 00:00 2024-04-25T11:36:07.293 INFO:teuthology.orchestra.run.smithi040.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 3.9 MB/s | 21 MB 00:05 2024-04-25T11:36:07.359 INFO:teuthology.orchestra.run.smithi062.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 929 kB/s | 85 kB 00:00 2024-04-25T11:36:07.409 INFO:teuthology.orchestra.run.smithi040.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 216 kB/s | 25 kB 00:00 2024-04-25T11:36:07.460 INFO:teuthology.orchestra.run.smithi062.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 170 kB 00:00 2024-04-25T11:36:07.602 INFO:teuthology.orchestra.run.smithi062.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 241 kB/s | 34 kB 00:00 2024-04-25T11:36:07.694 INFO:teuthology.orchestra.run.smithi062.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-25T11:36:07.936 INFO:teuthology.orchestra.run.smithi062.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 4.8 MB/s | 3.1 MB 00:00 2024-04-25T11:36:08.295 INFO:teuthology.orchestra.run.smithi062.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 5.9 MB/s | 3.5 MB 00:00 2024-04-25T11:36:08.403 INFO:teuthology.orchestra.run.smithi062.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.2 MB/s | 249 kB 00:00 2024-04-25T11:36:08.409 INFO:teuthology.orchestra.run.smithi040.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 4.3 MB/s | 17 MB 00:03 2024-04-25T11:36:08.515 INFO:teuthology.orchestra.run.smithi062.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 443 kB/s | 49 kB 00:00 2024-04-25T11:36:08.585 INFO:teuthology.orchestra.run.smithi040.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.3 MB/s | 225 kB 00:00 2024-04-25T11:36:08.612 INFO:teuthology.orchestra.run.smithi062.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 170 kB/s | 16 kB 00:00 2024-04-25T11:36:08.693 INFO:teuthology.orchestra.run.smithi040.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 250 kB/s | 27 kB 00:00 2024-04-25T11:36:08.729 INFO:teuthology.orchestra.run.smithi062.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 273 kB 00:00 2024-04-25T11:36:08.802 INFO:teuthology.orchestra.run.smithi040.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 293 kB/s | 32 kB 00:00 2024-04-25T11:36:08.929 INFO:teuthology.orchestra.run.smithi062.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.7 MB/s | 762 kB 00:00 2024-04-25T11:36:09.021 INFO:teuthology.orchestra.run.smithi062.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 199 kB/s | 18 kB 00:00 2024-04-25T11:36:09.069 INFO:teuthology.orchestra.run.smithi040.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.7 MB/s | 747 kB 00:00 2024-04-25T11:36:09.130 INFO:teuthology.orchestra.run.smithi062.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 744 kB/s | 80 kB 00:00 2024-04-25T11:36:09.186 INFO:teuthology.orchestra.run.smithi040.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 163 kB 00:00 2024-04-25T11:36:09.255 INFO:teuthology.orchestra.run.smithi062.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-25T11:36:09.287 INFO:teuthology.orchestra.run.smithi040.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.2 MB/s | 126 kB 00:00 2024-04-25T11:36:09.305 INFO:teuthology.orchestra.run.smithi062.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-25T11:36:09.356 INFO:teuthology.orchestra.run.smithi062.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T11:36:09.439 INFO:teuthology.orchestra.run.smithi062.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-25T11:36:09.470 INFO:teuthology.orchestra.run.smithi040.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 2.7 MB/s | 500 kB 00:00 2024-04-25T11:36:09.489 INFO:teuthology.orchestra.run.smithi062.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-25T11:36:09.540 INFO:teuthology.orchestra.run.smithi062.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T11:36:09.590 INFO:teuthology.orchestra.run.smithi062.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T11:36:09.641 INFO:teuthology.orchestra.run.smithi062.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T11:36:09.733 INFO:teuthology.orchestra.run.smithi062.stdout:(47/152): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-04-25T11:36:09.875 INFO:teuthology.orchestra.run.smithi062.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-04-25T11:36:09.958 INFO:teuthology.orchestra.run.smithi062.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-04-25T11:36:09.984 INFO:teuthology.orchestra.run.smithi062.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 983 kB/s | 107 kB 00:00 2024-04-25T11:36:10.051 INFO:teuthology.orchestra.run.smithi062.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 233 kB/s | 21 kB 00:00 2024-04-25T11:36:10.109 INFO:teuthology.orchestra.run.smithi062.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-04-25T11:36:10.151 INFO:teuthology.orchestra.run.smithi062.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-04-25T11:36:10.202 INFO:teuthology.orchestra.run.smithi062.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-25T11:36:10.252 INFO:teuthology.orchestra.run.smithi062.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-04-25T11:36:10.327 INFO:teuthology.orchestra.run.smithi062.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-25T11:36:10.345 INFO:teuthology.orchestra.run.smithi040.stdout:(18/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 2.2 MB/s | 11 MB 00:04 2024-04-25T11:36:10.411 INFO:teuthology.orchestra.run.smithi062.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 543 kB/s | 45 kB 00:00 2024-04-25T11:36:10.437 INFO:teuthology.orchestra.run.smithi040.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 493 kB/s | 45 kB 00:00 2024-04-25T11:36:10.538 INFO:teuthology.orchestra.run.smithi040.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.3 MB/s | 135 kB 00:00 2024-04-25T11:36:10.619 INFO:teuthology.orchestra.run.smithi062.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.9 MB/s | 3.0 MB 00:00 2024-04-25T11:36:10.644 INFO:teuthology.orchestra.run.smithi062.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-04-25T11:36:10.688 INFO:teuthology.orchestra.run.smithi040.stdout:(21/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 4.4 MB/s | 5.4 MB 00:01 2024-04-25T11:36:10.695 INFO:teuthology.orchestra.run.smithi062.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 510 kB/s | 25 kB 00:00 2024-04-25T11:36:10.713 INFO:teuthology.orchestra.run.smithi040.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 921 kB/s | 161 kB 00:00 2024-04-25T11:36:10.754 INFO:teuthology.orchestra.run.smithi062.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-25T11:36:10.787 INFO:teuthology.orchestra.run.smithi062.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-25T11:36:10.813 INFO:teuthology.orchestra.run.smithi040.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.5 MB/s | 320 kB 00:00 2024-04-25T11:36:10.862 INFO:teuthology.orchestra.run.smithi062.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-25T11:36:10.887 INFO:teuthology.orchestra.run.smithi062.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-25T11:36:10.922 INFO:teuthology.orchestra.run.smithi040.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.4 MB/s | 298 kB 00:00 2024-04-25T11:36:10.929 INFO:teuthology.orchestra.run.smithi062.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 528 kB/s | 35 kB 00:00 2024-04-25T11:36:10.947 INFO:teuthology.orchestra.run.smithi040.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 745 kB/s | 99 kB 00:00 2024-04-25T11:36:11.023 INFO:teuthology.orchestra.run.smithi040.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 851 kB/s | 85 kB 00:00 2024-04-25T11:36:11.139 INFO:teuthology.orchestra.run.smithi040.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.4 MB/s | 170 kB 00:00 2024-04-25T11:36:11.248 INFO:teuthology.orchestra.run.smithi040.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 316 kB/s | 34 kB 00:00 2024-04-25T11:36:11.280 INFO:teuthology.orchestra.run.smithi062.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-25T11:36:11.365 INFO:teuthology.orchestra.run.smithi040.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.2 MB/s | 142 kB 00:00 2024-04-25T11:36:11.489 INFO:teuthology.orchestra.run.smithi062.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-25T11:36:11.639 INFO:teuthology.orchestra.run.smithi062.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-04-25T11:36:11.697 INFO:teuthology.orchestra.run.smithi062.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.3 MB/s | 911 kB 00:00 2024-04-25T11:36:11.739 INFO:teuthology.orchestra.run.smithi062.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-25T11:36:11.757 INFO:teuthology.orchestra.run.smithi040.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.8 MB/s | 3.1 MB 00:00 2024-04-25T11:36:11.906 INFO:teuthology.orchestra.run.smithi062.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-25T11:36:11.998 INFO:teuthology.orchestra.run.smithi062.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 955 kB/s | 87 kB 00:00 2024-04-25T11:36:12.176 INFO:teuthology.orchestra.run.smithi062.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-04-25T11:36:12.276 INFO:teuthology.orchestra.run.smithi062.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-25T11:36:12.376 INFO:teuthology.orchestra.run.smithi062.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-25T11:36:12.426 INFO:teuthology.orchestra.run.smithi062.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-25T11:36:12.493 INFO:teuthology.orchestra.run.smithi062.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-04-25T11:36:12.569 INFO:teuthology.orchestra.run.smithi062.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T11:36:12.636 INFO:teuthology.orchestra.run.smithi062.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 521 kB/s | 35 kB 00:00 2024-04-25T11:36:12.890 INFO:teuthology.orchestra.run.smithi040.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.3 MB/s | 3.5 MB 00:01 2024-04-25T11:36:12.969 INFO:teuthology.orchestra.run.smithi062.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T11:36:13.032 INFO:teuthology.orchestra.run.smithi040.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.7 MB/s | 249 kB 00:00 2024-04-25T11:36:13.061 INFO:teuthology.orchestra.run.smithi062.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 5.6 MB/s | 6.0 MB 00:01 2024-04-25T11:36:13.108 INFO:teuthology.orchestra.run.smithi040.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 658 kB/s | 49 kB 00:00 2024-04-25T11:36:13.120 INFO:teuthology.orchestra.run.smithi062.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-25T11:36:13.187 INFO:teuthology.orchestra.run.smithi062.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-04-25T11:36:13.216 INFO:teuthology.orchestra.run.smithi040.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-04-25T11:36:13.229 INFO:teuthology.orchestra.run.smithi062.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 711 kB/s | 77 kB 00:00 2024-04-25T11:36:13.241 INFO:teuthology.orchestra.run.smithi040.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 123 kB/s | 16 kB 00:00 2024-04-25T11:36:13.288 INFO:teuthology.orchestra.run.smithi062.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-04-25T11:36:13.300 INFO:teuthology.orchestra.run.smithi040.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 273 kB 00:00 2024-04-25T11:36:13.363 INFO:teuthology.orchestra.run.smithi062.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-04-25T11:36:13.375 INFO:teuthology.orchestra.run.smithi040.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 243 kB/s | 18 kB 00:00 2024-04-25T11:36:13.388 INFO:teuthology.orchestra.run.smithi062.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-25T11:36:13.455 INFO:teuthology.orchestra.run.smithi062.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 765 kB/s | 51 kB 00:00 2024-04-25T11:36:13.480 INFO:teuthology.orchestra.run.smithi062.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-25T11:36:13.547 INFO:teuthology.orchestra.run.smithi062.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 585 kB/s | 54 kB 00:00 2024-04-25T11:36:13.568 INFO:teuthology.orchestra.run.smithi040.stdout:(38/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.3 MB/s | 762 kB 00:00 2024-04-25T11:36:13.593 INFO:teuthology.orchestra.run.smithi040.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 370 kB/s | 80 kB 00:00 2024-04-25T11:36:13.606 INFO:teuthology.orchestra.run.smithi062.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-04-25T11:36:13.690 INFO:teuthology.orchestra.run.smithi062.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 529 kB/s | 44 kB 00:00 2024-04-25T11:36:13.693 INFO:teuthology.orchestra.run.smithi040.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-25T11:36:13.790 INFO:teuthology.orchestra.run.smithi062.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-04-25T11:36:13.818 INFO:teuthology.orchestra.run.smithi040.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-25T11:36:13.848 INFO:teuthology.orchestra.run.smithi062.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-04-25T11:36:13.915 INFO:teuthology.orchestra.run.smithi062.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 713 kB/s | 47 kB 00:00 2024-04-25T11:36:13.927 INFO:teuthology.orchestra.run.smithi040.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-04-25T11:36:13.994 INFO:teuthology.orchestra.run.smithi040.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T11:36:14.049 INFO:teuthology.orchestra.run.smithi062.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-04-25T11:36:14.061 INFO:teuthology.orchestra.run.smithi040.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-25T11:36:14.128 INFO:teuthology.orchestra.run.smithi040.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T11:36:14.132 INFO:teuthology.orchestra.run.smithi062.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 694 kB/s | 58 kB 00:00 2024-04-25T11:36:14.195 INFO:teuthology.orchestra.run.smithi040.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-04-25T11:36:14.208 INFO:teuthology.orchestra.run.smithi062.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-25T11:36:14.220 INFO:teuthology.orchestra.run.smithi040.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T11:36:14.275 INFO:teuthology.orchestra.run.smithi062.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 958 kB/s | 64 kB 00:00 2024-04-25T11:36:14.334 INFO:teuthology.orchestra.run.smithi062.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-04-25T11:36:14.354 INFO:teuthology.orchestra.run.smithi040.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi040.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T11:36:14.463 INFO:teuthology.orchestra.run.smithi040.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-25T11:36:14.488 INFO:teuthology.orchestra.run.smithi040.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 287 kB/s | 21 kB 00:00 2024-04-25T11:36:14.547 INFO:teuthology.orchestra.run.smithi040.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-04-25T11:36:14.572 INFO:teuthology.orchestra.run.smithi040.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-25T11:36:14.609 INFO:teuthology.orchestra.run.smithi062.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-25T11:36:14.631 INFO:teuthology.orchestra.run.smithi040.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-25T11:36:14.634 INFO:teuthology.orchestra.run.smithi062.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2024-04-25T11:36:14.689 INFO:teuthology.orchestra.run.smithi040.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-25T11:36:14.748 INFO:teuthology.orchestra.run.smithi040.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T11:36:14.815 INFO:teuthology.orchestra.run.smithi040.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 681 kB/s | 45 kB 00:00 2024-04-25T11:36:14.968 INFO:teuthology.orchestra.run.smithi062.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-04-25T11:36:14.990 INFO:teuthology.orchestra.run.smithi040.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-04-25T11:36:15.074 INFO:teuthology.orchestra.run.smithi040.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-04-25T11:36:15.132 INFO:teuthology.orchestra.run.smithi040.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-04-25T11:36:15.158 INFO:teuthology.orchestra.run.smithi040.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-04-25T11:36:15.191 INFO:teuthology.orchestra.run.smithi040.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-25T11:36:15.217 INFO:teuthology.orchestra.run.smithi040.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T11:36:15.250 INFO:teuthology.orchestra.run.smithi040.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-25T11:36:15.275 INFO:teuthology.orchestra.run.smithi040.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-04-25T11:36:15.659 INFO:teuthology.orchestra.run.smithi040.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T11:36:15.793 INFO:teuthology.orchestra.run.smithi040.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.7 MB/s | 4.7 MB 00:00 2024-04-25T11:36:15.793 INFO:teuthology.orchestra.run.smithi062.stdout:(104/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 4.6 MB/s | 49 MB 00:10 2024-04-25T11:36:15.827 INFO:teuthology.orchestra.run.smithi062.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 360 kB/s | 309 kB 00:00 2024-04-25T11:36:15.834 INFO:teuthology.orchestra.run.smithi040.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-25T11:36:15.852 INFO:teuthology.orchestra.run.smithi062.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 450 kB/s | 548 kB 00:01 2024-04-25T11:36:15.969 INFO:teuthology.orchestra.run.smithi062.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-04-25T11:36:16.094 INFO:teuthology.orchestra.run.smithi062.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-04-25T11:36:16.119 INFO:teuthology.orchestra.run.smithi062.stdout:(109/152): grpc-data-1.46.7-10.el9.noarch.rpm 60 kB/s | 19 kB 00:00 2024-04-25T11:36:16.220 INFO:teuthology.orchestra.run.smithi062.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 537 kB/s | 67 kB 00:00 2024-04-25T11:36:16.293 INFO:teuthology.orchestra.run.smithi040.stdout:(69/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 5.5 MB/s | 49 MB 00:08 2024-04-25T11:36:16.318 INFO:teuthology.orchestra.run.smithi040.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 275 kB/s | 133 kB 00:00 2024-04-25T11:36:16.352 INFO:teuthology.orchestra.run.smithi040.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 1.6 MB/s | 911 kB 00:00 2024-04-25T11:36:16.411 INFO:teuthology.orchestra.run.smithi040.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T11:36:16.445 INFO:teuthology.orchestra.run.smithi062.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 465 kB/s | 151 kB 00:00 2024-04-25T11:36:16.519 INFO:teuthology.orchestra.run.smithi040.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-04-25T11:36:16.603 INFO:teuthology.orchestra.run.smithi040.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-25T11:36:16.662 INFO:teuthology.orchestra.run.smithi062.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.9 MB/s | 838 kB 00:00 2024-04-25T11:36:16.670 INFO:teuthology.orchestra.run.smithi040.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-25T11:36:16.720 INFO:teuthology.orchestra.run.smithi040.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-04-25T11:36:16.779 INFO:teuthology.orchestra.run.smithi040.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-04-25T11:36:16.795 INFO:teuthology.orchestra.run.smithi062.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 326 kB/s | 43 kB 00:00 2024-04-25T11:36:16.862 INFO:teuthology.orchestra.run.smithi040.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-25T11:36:16.896 INFO:teuthology.orchestra.run.smithi062.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 1.2 MB/s | 548 kB 00:00 2024-04-25T11:36:16.913 INFO:teuthology.orchestra.run.smithi040.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-04-25T11:36:16.921 INFO:teuthology.orchestra.run.smithi062.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-04-25T11:36:17.038 INFO:teuthology.orchestra.run.smithi062.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2024-04-25T11:36:17.063 INFO:teuthology.orchestra.run.smithi062.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-25T11:36:17.313 INFO:teuthology.orchestra.run.smithi040.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-25T11:36:17.355 INFO:teuthology.orchestra.run.smithi062.stdout:(118/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-04-25T11:36:17.381 INFO:teuthology.orchestra.run.smithi062.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.0 MB/s | 358 kB 00:00 2024-04-25T11:36:17.388 INFO:teuthology.orchestra.run.smithi040.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.1 MB/s | 6.0 MB 00:00 2024-04-25T11:36:17.406 INFO:teuthology.orchestra.run.smithi062.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 886 kB/s | 303 kB 00:00 2024-04-25T11:36:17.413 INFO:teuthology.orchestra.run.smithi040.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-04-25T11:36:17.447 INFO:teuthology.orchestra.run.smithi040.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T11:36:17.472 INFO:teuthology.orchestra.run.smithi040.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T11:36:17.489 INFO:teuthology.orchestra.run.smithi062.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 264 kB/s | 35 kB 00:00 2024-04-25T11:36:17.531 INFO:teuthology.orchestra.run.smithi040.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 505 kB/s | 42 kB 00:00 2024-04-25T11:36:17.556 INFO:teuthology.orchestra.run.smithi062.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-04-25T11:36:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-25T11:36:17.617 INFO:teuthology.orchestra.run.smithi040.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-25T11:36:17.623 INFO:teuthology.orchestra.run.smithi062.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T11:36:17.640 INFO:teuthology.orchestra.run.smithi040.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-25T11:36:17.682 INFO:teuthology.orchestra.run.smithi062.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-04-25T11:36:17.682 INFO:teuthology.orchestra.run.smithi040.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 764 kB/s | 51 kB 00:00 2024-04-25T11:36:17.716 INFO:teuthology.orchestra.run.smithi040.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 714 kB/s | 54 kB 00:00 2024-04-25T11:36:17.749 INFO:teuthology.orchestra.run.smithi062.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-04-25T11:36:17.783 INFO:teuthology.orchestra.run.smithi040.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-04-25T11:36:17.807 INFO:teuthology.orchestra.run.smithi062.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-04-25T11:36:17.841 INFO:teuthology.orchestra.run.smithi040.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-04-25T11:36:17.866 INFO:teuthology.orchestra.run.smithi062.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-04-25T11:36:17.900 INFO:teuthology.orchestra.run.smithi040.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T11:36:17.916 INFO:teuthology.orchestra.run.smithi062.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-25T11:36:17.959 INFO:teuthology.orchestra.run.smithi040.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-04-25T11:36:18.017 INFO:teuthology.orchestra.run.smithi040.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 815 kB/s | 47 kB 00:00 2024-04-25T11:36:18.033 INFO:teuthology.orchestra.run.smithi062.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-04-25T11:36:18.084 INFO:teuthology.orchestra.run.smithi040.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-25T11:36:18.143 INFO:teuthology.orchestra.run.smithi040.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-04-25T11:36:18.159 INFO:teuthology.orchestra.run.smithi062.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 629 kB/s | 79 kB 00:00 2024-04-25T11:36:18.193 INFO:teuthology.orchestra.run.smithi062.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 3.1 MB/s | 1.0 MB 00:00 2024-04-25T11:36:18.201 INFO:teuthology.orchestra.run.smithi040.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T11:36:18.252 INFO:teuthology.orchestra.run.smithi040.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T11:36:18.276 INFO:teuthology.orchestra.run.smithi062.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 493 kB/s | 58 kB 00:00 2024-04-25T11:36:18.312 INFO:teuthology.orchestra.run.smithi040.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 743 kB/s | 45 kB 00:00 2024-04-25T11:36:18.318 INFO:teuthology.orchestra.run.smithi062.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-25T11:36:18.388 INFO:teuthology.orchestra.run.smithi040.stdout:(101/152): postgresql-server-13.14-1.el9.x86_64 2.8 MB/s | 5.8 MB 00:02 2024-04-25T11:36:18.393 INFO:teuthology.orchestra.run.smithi062.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-25T11:36:18.435 INFO:teuthology.orchestra.run.smithi062.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 769 kB/s | 90 kB 00:00 2024-04-25T11:36:18.463 INFO:teuthology.orchestra.run.smithi040.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-04-25T11:36:18.511 INFO:teuthology.orchestra.run.smithi062.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-04-25T11:36:18.553 INFO:teuthology.orchestra.run.smithi062.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-04-25T11:36:18.644 INFO:teuthology.orchestra.run.smithi062.stdout:(138/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.6 MB/s | 2.0 MB 00:01 2024-04-25T11:36:18.670 INFO:teuthology.orchestra.run.smithi062.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-25T11:36:18.695 INFO:teuthology.orchestra.run.smithi062.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 416 kB/s | 59 kB 00:00 2024-04-25T11:36:18.787 INFO:teuthology.orchestra.run.smithi062.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-04-25T11:36:18.812 INFO:teuthology.orchestra.run.smithi062.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 359 kB/s | 51 kB 00:00 2024-04-25T11:36:18.857 INFO:teuthology.orchestra.run.smithi062.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-04-25T11:36:18.896 INFO:teuthology.orchestra.run.smithi062.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-04-25T11:36:18.971 INFO:teuthology.orchestra.run.smithi062.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 175 kB/s | 20 kB 00:00 2024-04-25T11:36:19.013 INFO:teuthology.orchestra.run.smithi062.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T11:36:19.047 INFO:teuthology.orchestra.run.smithi062.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.8 MB/s | 427 kB 00:00 2024-04-25T11:36:19.121 INFO:teuthology.orchestra.run.smithi040.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-04-25T11:36:19.172 INFO:teuthology.orchestra.run.smithi040.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-04-25T11:36:19.280 INFO:teuthology.orchestra.run.smithi062.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-04-25T11:36:19.439 INFO:teuthology.orchestra.run.smithi040.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-25T11:36:19.506 INFO:teuthology.orchestra.run.smithi062.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 1.3 MB 00:00 2024-04-25T11:36:19.548 INFO:teuthology.orchestra.run.smithi040.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 506 kB/s | 548 kB 00:01 2024-04-25T11:36:19.614 INFO:teuthology.orchestra.run.smithi062.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-25T11:36:19.656 INFO:teuthology.orchestra.run.smithi040.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 578 kB/s | 309 kB 00:00 2024-04-25T11:36:19.764 INFO:teuthology.orchestra.run.smithi062.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.4 MB/s | 3.1 MB 00:00 2024-04-25T11:36:19.823 INFO:teuthology.orchestra.run.smithi062.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 4.2 MB/s | 3.4 MB 00:00 2024-04-25T11:36:19.825 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:36:19.826 INFO:teuthology.orchestra.run.smithi062.stdout:Total 11 MB/s | 222 MB 00:19 2024-04-25T11:36:19.839 INFO:teuthology.orchestra.run.smithi040.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 89 kB/s | 26 kB 00:00 2024-04-25T11:36:19.998 INFO:teuthology.orchestra.run.smithi040.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-25T11:36:20.182 INFO:teuthology.orchestra.run.smithi040.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-04-25T11:36:20.424 INFO:teuthology.orchestra.run.smithi040.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-25T11:36:20.549 INFO:teuthology.orchestra.run.smithi040.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-04-25T11:36:20.858 INFO:teuthology.orchestra.run.smithi040.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-25T11:36:20.891 INFO:teuthology.orchestra.run.smithi040.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-04-25T11:36:20.923 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-04-25T11:36:21.067 INFO:teuthology.orchestra.run.smithi040.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 853 kB/s | 548 kB 00:00 2024-04-25T11:36:21.083 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-04-25T11:36:21.083 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-04-25T11:36:21.125 INFO:teuthology.orchestra.run.smithi040.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 120 kB/s | 32 kB 00:00 2024-04-25T11:36:21.159 INFO:teuthology.orchestra.run.smithi040.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-25T11:36:21.501 INFO:teuthology.orchestra.run.smithi040.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 396 kB/s | 172 kB 00:00 2024-04-25T11:36:21.634 INFO:teuthology.orchestra.run.smithi040.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-04-25T11:36:21.667 INFO:teuthology.orchestra.run.smithi040.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 596 kB/s | 303 kB 00:00 2024-04-25T11:36:21.768 INFO:teuthology.orchestra.run.smithi040.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-25T11:36:22.143 INFO:teuthology.orchestra.run.smithi040.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 404 kB/s | 206 kB 00:00 2024-04-25T11:36:22.193 INFO:teuthology.orchestra.run.smithi040.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-25T11:36:22.327 INFO:teuthology.orchestra.run.smithi040.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-04-25T11:36:22.452 INFO:teuthology.orchestra.run.smithi040.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 68 kB/s | 18 kB 00:00 2024-04-25T11:36:22.545 INFO:teuthology.orchestra.run.smithi040.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.3 MB/s | 2.0 MB 00:00 2024-04-25T11:36:22.586 INFO:teuthology.orchestra.run.smithi040.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 89 kB/s | 23 kB 00:00 2024-04-25T11:36:22.720 INFO:teuthology.orchestra.run.smithi040.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-25T11:36:22.804 INFO:teuthology.orchestra.run.smithi040.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 76 kB/s | 20 kB 00:00 2024-04-25T11:36:22.972 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-04-25T11:36:22.972 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-04-25T11:36:23.071 INFO:teuthology.orchestra.run.smithi040.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-04-25T11:36:23.146 INFO:teuthology.orchestra.run.smithi040.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 231 kB/s | 79 kB 00:00 2024-04-25T11:36:23.263 INFO:teuthology.orchestra.run.smithi040.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-25T11:36:23.413 INFO:teuthology.orchestra.run.smithi040.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 169 kB/s | 58 kB 00:00 2024-04-25T11:36:23.530 INFO:teuthology.orchestra.run.smithi040.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-04-25T11:36:23.655 INFO:teuthology.orchestra.run.smithi040.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-04-25T11:36:23.797 INFO:teuthology.orchestra.run.smithi040.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 106 kB/s | 28 kB 00:00 2024-04-25T11:36:23.839 INFO:teuthology.orchestra.run.smithi040.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 211 kB/s | 90 kB 00:00 2024-04-25T11:36:23.914 INFO:teuthology.orchestra.run.smithi040.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-04-25T11:36:24.173 INFO:teuthology.orchestra.run.smithi040.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 138 kB/s | 36 kB 00:00 2024-04-25T11:36:24.198 INFO:teuthology.orchestra.run.smithi040.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 165 kB/s | 59 kB 00:00 2024-04-25T11:36:24.223 INFO:teuthology.orchestra.run.smithi040.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 442 kB/s | 188 kB 00:00 2024-04-25T11:36:24.524 INFO:teuthology.orchestra.run.smithi040.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 145 kB/s | 51 kB 00:00 2024-04-25T11:36:24.658 INFO:teuthology.orchestra.run.smithi040.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 207 kB/s | 90 kB 00:00 2024-04-25T11:36:24.708 INFO:teuthology.orchestra.run.smithi040.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-04-25T11:36:24.925 INFO:teuthology.orchestra.run.smithi040.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-25T11:36:25.108 INFO:teuthology.orchestra.run.smithi040.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-04-25T11:36:25.133 INFO:teuthology.orchestra.run.smithi040.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-04-25T11:36:25.683 INFO:teuthology.orchestra.run.smithi040.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-25T11:36:25.959 INFO:teuthology.orchestra.run.smithi040.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 1.3 MB 00:00 2024-04-25T11:36:26.151 INFO:teuthology.orchestra.run.smithi040.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-04-25T11:36:26.384 INFO:teuthology.orchestra.run.smithi040.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.5 MB/s | 3.1 MB 00:01 2024-04-25T11:36:26.576 INFO:teuthology.orchestra.run.smithi040.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 2.3 MB/s | 3.4 MB 00:01 2024-04-25T11:36:26.579 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:36:26.579 INFO:teuthology.orchestra.run.smithi040.stdout:Total 8.7 MB/s | 222 MB 00:25 2024-04-25T11:36:27.728 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-25T11:36:27.885 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-25T11:36:27.885 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-25T11:36:29.178 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-04-25T11:36:29.286 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:36:29.519 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:36:29.599 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:36:29.646 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:36:29.679 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:36:29.724 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:36:29.755 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-25T11:36:29.756 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-25T11:36:30.067 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:36:30.189 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:36:30.320 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:36:30.347 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:36:30.439 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:36:30.468 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:36:30.624 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:36:30.674 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:36:30.702 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:36:30.790 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:36:30.819 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:36:30.865 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:36:30.887 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:36:30.907 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:36:31.113 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:36:31.145 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:36:31.256 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:36:31.351 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:36:31.386 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:36:31.421 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:36:31.496 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:36:31.547 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:36:31.608 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:36:31.646 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:36:31.674 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:36:31.746 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:36:31.805 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:36:31.834 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:36:31.870 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:36:31.949 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:36:31.981 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T11:36:32.024 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T11:36:32.053 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T11:36:32.160 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:36:32.352 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:36:32.438 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:36:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:36:32.655 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:36:33.209 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:36:33.302 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:36:33.336 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:36:33.379 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:36:33.417 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T11:36:33.459 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:36:33.649 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:36:33.711 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:36:33.768 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:36:33.842 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:36:33.879 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:36:33.915 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:36:33.963 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:36:33.998 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:36:34.042 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:36:34.084 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:36:34.236 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:36:34.289 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:36:34.319 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:36:34.361 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:36:34.459 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:36:34.494 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:36:34.534 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:36:34.567 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:36:34.592 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:36:34.678 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:36:34.817 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:36:34.866 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:36:34.930 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:36:34.930 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:36:34.967 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:36:34.999 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:36:35.029 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:36:35.063 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:36:35.104 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:36:35.155 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:36:35.189 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:36:35.280 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:36:35.329 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:36:35.359 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:36:35.407 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:36:35.517 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:36:35.578 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:36:35.788 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-25T11:36:35.880 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:36:35.919 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:36:35.981 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:36:36.023 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:36:36.048 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:36:36.091 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:36:36.343 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:36:36.399 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:36:36.433 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:36:36.436 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:36:36.535 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:36:36.540 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:36:36.600 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:36:36.665 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:36:36.692 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:36:36.692 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:36:36.727 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:36:36.784 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:36:36.813 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:36:36.911 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:36:36.940 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:36:36.973 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:36:37.020 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:36:37.027 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:36:37.074 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:36:37.168 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:36:37.197 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:36:37.260 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:36:37.281 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:36:37.307 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:36:37.499 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:36:37.531 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:36:37.642 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:36:37.705 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:36:37.738 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:36:37.772 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:36:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:36:37.816 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:36:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:36:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:36:37.968 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:36:38.008 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:36:38.038 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:36:38.125 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:36:38.184 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:36:38.213 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:36:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:36:38.337 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:36:38.376 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T11:36:38.428 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T11:36:38.457 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T11:36:38.531 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:36:38.580 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:36:38.657 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:36:38.793 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:36:38.859 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:36:39.119 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:36:39.194 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:36:39.222 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:36:39.255 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:36:39.413 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:36:39.498 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:36:39.540 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:36:39.582 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:36:39.583 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:36:39.621 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T11:36:39.626 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:36:39.663 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:36:39.709 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:36:39.739 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T11:36:39.776 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:36:39.823 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:36:39.874 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:36:39.938 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:36:40.013 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:36:40.049 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:36:40.078 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:36:40.125 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:36:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:36:40.213 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:36:40.254 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:36:40.279 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:36:40.304 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:36:40.358 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:36:40.397 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T11:36:40.399 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:36:40.443 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:36:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:36:40.515 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:36:40.615 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:36:40.656 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:36:40.696 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:36:40.721 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:36:40.746 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:36:40.833 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:36:40.962 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:36:40.996 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:36:41.024 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:36:41.058 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:36:41.097 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:36:41.137 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:36:41.183 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:36:41.218 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:36:41.260 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:36:41.302 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:36:41.336 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:36:41.427 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:36:41.475 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:36:41.506 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:36:41.546 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:36:41.658 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:36:41.708 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:36:42.481 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:36:42.538 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:36:42.572 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:36:42.683 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:36:42.739 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:36:42.806 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:36:42.829 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:36:42.843 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:36:42.865 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:36:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:36:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:36:43.151 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:36:43.411 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:36:43.446 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:36:43.537 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:36:43.537 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:36:43.538 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:36:43.538 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:36:43.564 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:36:43.845 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:36:43.958 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:36:45.258 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:36:45.325 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:36:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:36:45.386 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:36:45.716 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:36:45.757 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:36:45.832 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:36:45.862 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T11:36:45.899 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:36:46.403 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:36:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:36:46.507 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:36:46.537 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T11:36:48.971 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:36:49.008 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:36:49.580 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:36:49.620 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:36:49.713 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:36:49.713 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:36:49.713 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:36:49.713 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:36:49.747 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:36:55.452 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:36:55.453 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-04-25T11:36:55.453 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-04-25T11:36:55.453 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-04-25T11:36:55.453 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-04-25T11:36:55.453 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-04-25T11:36:55.453 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-04-25T11:36:55.453 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-04-25T11:36:55.453 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:36:55.633 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:36:55.649 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:36:55.649 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:36:55.650 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:36:55.650 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:36:57.177 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:36:57.204 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:36:57.205 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:36:57.205 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:36:57.205 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:36:57.205 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:36:57.205 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:36:57.436 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:36:57.459 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:36:57.459 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:36:57.459 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:36:57.459 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:36:57.459 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:36:57.459 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:36:57.920 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:36:57.944 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:36:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:36:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:36:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:36:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:36:57.944 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:36:58.051 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:36:58.110 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:36:58.173 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:36:58.369 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:36:58.430 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:36:58.510 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:36:58.609 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:36:58.752 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:36:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:36:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:36:58.847 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:36:59.308 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:36:59.345 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:36:59.378 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T11:36:59.441 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T11:36:59.468 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:37:00.564 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:37:00.604 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:37:01.620 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:37:01.645 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:37:01.755 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:37:01.902 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:37:01.902 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-04-25T11:37:01.902 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-04-25T11:37:01.902 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-04-25T11:37:01.902 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-04-25T11:37:01.902 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-04-25T11:37:01.902 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-04-25T11:37:01.902 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-04-25T11:37:01.903 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:01.925 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T11:37:01.956 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:37:01.980 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:37:01.980 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:01.980 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:37:01.980 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:37:01.981 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:37:01.981 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:02.023 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:37:02.034 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:37:02.084 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:37:02.101 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:37:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:02.101 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:37:02.101 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:02.136 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T11:37:02.189 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T11:37:03.163 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:37:03.201 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:37:03.224 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:37:03.224 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:03.224 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:37:03.225 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:37:03.225 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:37:03.225 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:03.268 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:37:03.291 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:37:03.291 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:03.291 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:37:03.291 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:03.335 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:37:03.358 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:37:03.358 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:03.358 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:37:03.358 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:37:03.359 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:37:03.359 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:03.662 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:37:03.677 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:37:03.690 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:37:03.690 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:03.690 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:37:03.690 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:37:03.690 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:37:03.690 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:03.701 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:37:03.701 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:03.701 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:37:03.701 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:37:03.701 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:37:03.701 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:03.929 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:37:03.953 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:37:03.953 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:03.953 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:37:03.953 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:37:03.953 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:37:03.953 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:04.413 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:37:04.437 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:37:04.437 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:04.437 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:37:04.437 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:37:04.437 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:37:04.437 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:04.544 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:37:04.602 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:37:04.666 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:37:04.859 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:37:04.906 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:37:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:37:05.075 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:37:05.242 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:37:05.281 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:37:05.314 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:37:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:37:05.812 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:37:05.839 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:37:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T11:37:05.936 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T11:37:05.971 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:37:07.068 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:37:07.106 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:37:08.124 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:37:08.148 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:37:08.258 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:37:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T11:37:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:37:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:37:08.483 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:08.483 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:37:08.483 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:37:08.483 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:37:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:08.534 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:37:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:37:08.556 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T11:37:08.598 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T11:37:08.638 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T11:37:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T11:37:08.670 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:37:08.697 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:37:08.699 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T11:37:08.890 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T11:37:08.935 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:37:08.964 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T11:37:09.561 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:37:09.603 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:37:09.624 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:37:09.676 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:37:09.712 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:37:09.736 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:37:09.736 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:09.736 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:37:09.736 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:37:09.736 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:37:09.736 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:09.779 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:37:09.805 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:37:09.805 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:09.806 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:37:09.806 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:09.854 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:37:09.879 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:37:09.879 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:09.879 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:37:09.879 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:37:09.879 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:37:09.879 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:10.205 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:37:10.229 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:37:10.229 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:37:10.229 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:37:10.229 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:37:10.230 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:37:10.230 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:10.542 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T11:37:10.658 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:37:10.700 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:37:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:37:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:37:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:37:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:37:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:37:10.954 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:37:11.044 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:37:11.114 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:37:15.092 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T11:37:15.134 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T11:37:15.183 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T11:37:15.215 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:37:15.242 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:37:15.439 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T11:37:15.488 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:37:15.518 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T11:37:16.153 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:37:16.191 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:37:16.211 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:37:17.110 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T11:37:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:37:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T11:37:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T11:37:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T11:37:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T11:37:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T11:37:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T11:37:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T11:37:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T11:37:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T11:37:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T11:37:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:37:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:37:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:37:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:37:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:37:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:37:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:37:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:37:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:37:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:37:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:37:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T11:37:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:37:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T11:37:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:37:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:37:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:37:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:37:17.195 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:37:17.237 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:37:17.323 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:37:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:37:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:37:17.459 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:37:17.503 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:37:17.503 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:37:17.583 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:37:17.651 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:37:19.252 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:37:19.252 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:19.252 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-04-25T11:37:19.252 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.253 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.254 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:37:19.255 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.256 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:37:19.257 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:37:19.258 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:37:19.259 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:37:19.260 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:37:19.261 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:37:19.262 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:37:19.263 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:37:19.263 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:37:19.263 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:37:19.263 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:19.263 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-25T11:37:19.630 DEBUG:teuthology.parallel:result is None 2024-04-25T11:37:23.360 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T11:37:23.361 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T11:37:23.362 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T11:37:23.363 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T11:37:23.363 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T11:37:23.374 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T11:37:23.374 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T11:37:23.374 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T11:37:23.374 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T11:37:23.374 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T11:37:23.374 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T11:37:23.374 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T11:37:23.374 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:37:23.376 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:37:23.377 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:37:23.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:37:23.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:37:23.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:37:23.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:37:23.382 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:37:25.598 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:37:25.598 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:25.598 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-04-25T11:37:25.598 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.598 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.598 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:37:25.601 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:37:25.602 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:37:25.603 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:37:25.604 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:37:25.605 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:37:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:37:25.607 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:25.608 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-25T11:37:26.001 DEBUG:teuthology.parallel:result is None 2024-04-25T11:37:26.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:37:26.151 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:37:26.173 INFO:teuthology.orchestra.run.smithi040.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T11:37:26.174 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T11:37:26.174 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T11:37:26.175 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:37:26.311 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:37:26.338 INFO:teuthology.orchestra.run.smithi062.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T11:37:26.338 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T11:37:26.338 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T11:37:26.339 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T11:37:26.339 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:26.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:37:26.368 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:37:26.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:37:26.410 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T11:37:26.410 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:26.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:37:26.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:37:26.503 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:37:26.503 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:37:26.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:37:26.600 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T11:37:26.600 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:26.600 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:37:26.624 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:37:26.688 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:37:26.688 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:37:26.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:37:26.784 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T11:37:26.784 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:26.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:37:26.808 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:37:26.872 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:37:26.872 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:37:26.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:37:26.969 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T11:37:27.045 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T11:37:27.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2024-04-25T11:37:27.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2024-04-25T11:37:27.075 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T11:37:27.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T11:37:27.113 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T11:37:27.143 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T11:37:27.143 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T11:37:27.180 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T11:37:27.216 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T11:37:27.216 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T11:37:27.217 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T11:37:27.217 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:27.217 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T11:37:27.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:27.371 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:27.395 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:37:27.395 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T11:37:27.413 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:27.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:27.561 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T11:37:27.561 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'fuse set user groups': True, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T11:37:27.561 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332', 'branch': 'wip-lusov-quiescer', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/suites/pjd}}', 'email': None, 'first_in_suite': False, 'job_id': '7673332', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-quiescer'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'fuse set user groups': True, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-quiescer', 'sha1': 'df40303764f976ac2e973c1ef2b40d1ebbda473b'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 2866, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'sleep_before_teardown': 0, 'subset': '200/111111', 'suite': 'fs', 'suite_branch': 'wip-lusov-quiescer', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph', 'suite_sha1': 'df40303764f976ac2e973c1ef2b40d1ebbda473b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJNl3N2R7LyI0uZGFSlGW1HbkwyyfKdvKA0bEJ5k3GWtbtg5eNAT09UYtGbAozRjlBR7SLesr+YXP/WxnAt3IM=', 'smithi062.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJktnmc6KegBeiC88dl+vdB9DEb/41t0qgszN4lGnfkUjzaK+GY6tk2mzVIgxPl2AmEqcIYAtrV7imk9k1IzJ0k='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'fuse set user groups': True, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'quiescer': None}, {'workunit': {'clients': {'all': ['suites/pjd.sh']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_11:13:36', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T11:37:27.562 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T11:37:27.564 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T11:37:27.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T11:37:27.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T11:37:27.679 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:27.679 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:37:27.699 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:37:27.699 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-04-25T11:37:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:37:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:37:27.755 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T11:37:27.755 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:37:27.755 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:37:27.755 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 11:37:18.042704544 +0000 2024-04-25T11:37:27.755 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 11:35:18.659595978 +0000 2024-04-25T11:37:27.755 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 11:35:18.659595978 +0000 2024-04-25T11:37:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 11:35:18.659595978 +0000 2024-04-25T11:37:27.755 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:37:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T11:37:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T11:37:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000262087 s, 2.0 MB/s 2024-04-25T11:37:27.821 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:37:27.877 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-04-25T11:37:27.932 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:37:27.932 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:37:27.932 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T11:37:27.932 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:37:27.932 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:37:27.933 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 11:37:18.042704544 +0000 2024-04-25T11:37:27.933 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 11:35:19.004591418 +0000 2024-04-25T11:37:27.933 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 11:35:19.004591418 +0000 2024-04-25T11:37:27.933 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 11:35:19.004591418 +0000 2024-04-25T11:37:27.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:37:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T11:37:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T11:37:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000231021 s, 2.2 MB/s 2024-04-25T11:37:27.995 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:37:28.052 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-04-25T11:37:28.107 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:37:28.107 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:37:28.107 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T11:37:28.107 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:37:28.107 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:37:28.107 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 11:37:18.043704531 +0000 2024-04-25T11:37:28.107 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 11:35:19.357586752 +0000 2024-04-25T11:37:28.108 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 11:35:19.357586752 +0000 2024-04-25T11:37:28.108 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 11:35:19.357586752 +0000 2024-04-25T11:37:28.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:37:28.170 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T11:37:28.170 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T11:37:28.170 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000201097 s, 2.5 MB/s 2024-04-25T11:37:28.171 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:37:28.229 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-04-25T11:37:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:37:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:37:28.285 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T11:37:28.285 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:37:28.285 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:37:28.285 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 11:37:18.043704531 +0000 2024-04-25T11:37:28.285 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 11:35:19.707582126 +0000 2024-04-25T11:37:28.285 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 11:35:19.707582126 +0000 2024-04-25T11:37:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 11:35:19.707582126 +0000 2024-04-25T11:37:28.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:37:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T11:37:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T11:37:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000178984 s, 2.9 MB/s 2024-04-25T11:37:28.349 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:37:28.405 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:28.405 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:37:28.405 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:37:28.424 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:37:28.424 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-04-25T11:37:28.483 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:37:28.483 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:37:28.483 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T11:37:28.483 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:37:28.483 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:37:28.483 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 11:37:11.520511373 +0000 2024-04-25T11:37:28.483 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 11:35:18.665802740 +0000 2024-04-25T11:37:28.483 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 11:35:18.665802740 +0000 2024-04-25T11:37:28.483 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 11:35:18.665802740 +0000 2024-04-25T11:37:28.484 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:37:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T11:37:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T11:37:28.550 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000281929 s, 1.8 MB/s 2024-04-25T11:37:28.551 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:37:28.609 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-04-25T11:37:28.666 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:37:28.667 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:37:28.667 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T11:37:28.667 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:37:28.667 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:37:28.667 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 11:37:11.520511373 +0000 2024-04-25T11:37:28.667 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 11:35:19.015796666 +0000 2024-04-25T11:37:28.667 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 11:35:19.015796666 +0000 2024-04-25T11:37:28.667 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 11:35:19.015796666 +0000 2024-04-25T11:37:28.667 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:37:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T11:37:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T11:37:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000177862 s, 2.9 MB/s 2024-04-25T11:37:28.734 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:37:28.791 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-04-25T11:37:28.848 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:37:28.848 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:37:28.849 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T11:37:28.849 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:37:28.849 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:37:28.849 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 11:37:11.520511373 +0000 2024-04-25T11:37:28.849 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 11:35:19.357790731 +0000 2024-04-25T11:37:28.849 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 11:35:19.357790731 +0000 2024-04-25T11:37:28.849 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 11:35:19.357790731 +0000 2024-04-25T11:37:28.849 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:37:28.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T11:37:28.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T11:37:28.913 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00019352 s, 2.6 MB/s 2024-04-25T11:37:28.914 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:37:28.971 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-04-25T11:37:29.027 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:37:29.028 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:37:29.028 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T11:37:29.028 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:37:29.028 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:37:29.028 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 11:37:11.521511357 +0000 2024-04-25T11:37:29.028 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 11:35:19.703784727 +0000 2024-04-25T11:37:29.028 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 11:35:19.703784727 +0000 2024-04-25T11:37:29.028 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 11:35:19.703784727 +0000 2024-04-25T11:37:29.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:37:29.091 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T11:37:29.091 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T11:37:29.091 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000176589 s, 2.9 MB/s 2024-04-25T11:37:29.092 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:37:29.149 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:29.149 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:37:29.149 INFO:tasks.ceph:Generating config... 2024-04-25T11:37:29.150 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T11:37:29.150 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-25T11:37:29.150 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T11:37:29.150 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T11:37:29.151 INFO:tasks.ceph:[client] fuse set user groups = True 2024-04-25T11:37:29.151 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T11:37:29.151 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T11:37:29.151 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T11:37:29.152 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T11:37:29.152 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T11:37:29.152 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-25T11:37:29.152 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T11:37:29.152 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T11:37:29.153 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T11:37:29.153 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T11:37:29.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T11:37:29.217 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T11:37:29.220 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T11:37:29.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:37:29.287 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.62'), ('mon.c', '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]')] 2024-04-25T11:37:29.287 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'fuse set user groups': True, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T11:37:29.288 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:29.288 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:37:29.343 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.62 --addv c '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid 6d14dde7-764d-42d0-be37-c790a704858d 2024-04-25T11:37:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = quincy 2024-04-25T11:37:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2024-04-25T11:37:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:fsid 6d14dde7-764d-42d0-be37-c790a704858d 2024-04-25T11:37:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2024-04-25T11:37:29.442988+0000 2024-04-25T11:37:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-04-25T11:37:29.442988+0000 2024-04-25T11:37:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 17 (quincy) 2024-04-25T11:37:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2024-04-25T11:37:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-04-25T11:37:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2024-04-25T11:37:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] mon.c 2024-04-25T11:37:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T11:37:29.431 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:37:29.446 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6d14dde7-764d-42d0-be37-c790a704858d... 2024-04-25T11:37:29.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T11:37:29.488 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2024-04-25T11:37:29.528 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2024-04-25T11:37:29.529 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2024-04-25T11:37:29.530 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]" 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = 6d14dde7-764d-42d0-be37-c790a704858d 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2024-04-25T11:37:29.531 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug trim objects = true 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.532 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore compression mode = aggressive 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 20 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 20 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2024-04-25T11:37:29.533 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: osd blocked scrub grace period = 3600 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: osd max markdown count = 1000 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:37:29.534 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T11:37:29.535 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: mon down mkfs grace = 300 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: mon op complaint time = 120 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T11:37:29.536 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout: client mount timeout = 600 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout: client_shutdown_timeout = 120 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout: fuse set user groups = True 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.e] 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.f] 2024-04-25T11:37:29.537 INFO:teuthology.orchestra.run.smithi040.stdout:[mds] 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 20 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds balancer = 20 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal fragment size max = 10000 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal merge size = 5 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal split bits = 3 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal split size = 100 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug frag = True 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug scatterstat = True 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds oft prefetch dirfrags = True 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds op complaint time = 180 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds verify scatter = True 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds_dir_prefetch = True 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: mds_sleep_rank_change = 5000000.0 2024-04-25T11:37:29.538 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-04-25T11:37:29.539 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-04-25T11:37:29.539 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-04-25T11:37:29.539 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-04-25T11:37:29.539 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2024-04-25T11:37:29.539 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T11:37:29.539 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2024-04-25T11:37:29.539 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.539 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.540 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.541 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.40,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]" 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 6d14dde7-764d-42d0-be37-c790a704858d 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 600 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock profile = high_recovery_ops 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.542 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug trim objects = true 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.543 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore compression mode = aggressive 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 20 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 20 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 10 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: osd blocked scrub grace period = 3600 2024-04-25T11:37:29.544 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: osd max markdown count = 1000 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T11:37:29.545 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T11:37:29.546 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: mon down mkfs grace = 300 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: mon op complaint time = 120 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T11:37:29.547 INFO:teuthology.orchestra.run.smithi062.stdout: client mount timeout = 600 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout: client_shutdown_timeout = 120 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout: fuse set user groups = True 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.e] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.f] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout:[mds] 2024-04-25T11:37:29.548 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds = 20 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds balancer = 20 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds bal fragment size max = 10000 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds bal merge size = 5 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds bal split bits = 3 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds bal split size = 100 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug frag = True 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug scatterstat = True 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds oft prefetch dirfrags = True 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds op complaint time = 180 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds verify scatter = True 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds_dir_prefetch = True 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: mds_sleep_rank_change = 5000000.0 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2024-04-25T11:37:29.549 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2024-04-25T11:37:29.551 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T11:37:29.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T11:37:29.644 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T11:37:29.644 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:29.644 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T11:37:29.659 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:29.659 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T11:37:29.714 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:37:29.714 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:29.714 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T11:37:29.714 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:37:29.787 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:37:29.788 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:29.843 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:37:29.843 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:37:29.843 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T11:37:29.843 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:37:29.881 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:37:29.881 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:29.938 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T11:37:29.938 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T11:37:29.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:37:29.988 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:37:29.990 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:37:30.070 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:37:30.073 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T11:37:30.073 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T11:37:30.123 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T11:37:30.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T11:37:30.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T11:37:30.239 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T11:37:30.242 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T11:37:30.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-25T11:37:30.356 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-25T11:37:30.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-25T11:37:30.382 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T11:37:30.436 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T11:37:30.438 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T11:37:30.463 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T11:37:30.553 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T11:37:30.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T11:37:30.579 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-25T11:37:30.668 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-25T11:37:30.671 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-25T11:37:30.695 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T11:37:30.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T11:37:30.736 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T11:37:30.748 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T11:37:30.748 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:37:30.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T11:37:30.812 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:30.812 INFO:tasks.ceph:role: osd.0 2024-04-25T11:37:30.812 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:37:30.812 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T11:37:30.883 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:37:30.883 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:37:30.884 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:37:30.884 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:37:30.884 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:37:30.884 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:37:30.884 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:37:30.884 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:37:30.884 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:37:30.884 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:37:31.369 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-25T11:37:31.371 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-25T11:37:31.371 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T11:37:31.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T11:37:31.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T11:37:31.583 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:31.583 INFO:tasks.ceph:role: osd.1 2024-04-25T11:37:31.583 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:37:31.583 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T11:37:31.655 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:37:31.655 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:37:31.655 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:37:31.655 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:37:31.656 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:37:31.656 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:37:31.656 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:37:31.656 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:37:31.656 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:37:31.656 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:37:32.120 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-25T11:37:32.122 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-25T11:37:32.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T11:37:32.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T11:37:32.238 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T11:37:32.303 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:32.304 INFO:tasks.ceph:role: osd.2 2024-04-25T11:37:32.304 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:37:32.304 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T11:37:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:37:32.376 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:37:32.376 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:37:32.376 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:37:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:37:32.376 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:37:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:37:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:37:32.376 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:37:32.377 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:37:32.867 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-25T11:37:32.869 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-25T11:37:32.869 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T11:37:32.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T11:37:33.027 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T11:37:33.091 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:33.091 INFO:tasks.ceph:role: osd.3 2024-04-25T11:37:33.091 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:37:33.091 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T11:37:33.163 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:37:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:37:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:37:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:37:33.164 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:37:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:37:33.164 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:37:33.164 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:37:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:37:33.164 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:37:33.664 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-25T11:37:33.666 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-25T11:37:33.666 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T11:37:33.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T11:37:33.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:33.924+0000 7f684a33e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T11:37:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:33.924+0000 7f684a33e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T11:37:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:33.924+0000 7f684a33e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:37:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:33.924+0000 7f684a33e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:37:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:33.924+0000 7f684a33e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:37:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:33.925+0000 7f684a33e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T11:37:36.774 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T11:37:36.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:36.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:36.905+0000 7f0c52419640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T11:37:36.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:36.906+0000 7f0c52419640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T11:37:36.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:36.906+0000 7f0c52419640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:37:36.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:36.906+0000 7f0c52419640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:37:36.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:36.906+0000 7f0c52419640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:37:36.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:36.906+0000 7f0c52419640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T11:37:39.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T11:37:39.776 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:39.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:39.878+0000 7f46e8c0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T11:37:39.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:39.878+0000 7f46e8c0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T11:37:39.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:39.878+0000 7f46e8c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:37:39.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:39.878+0000 7f46e8c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:37:39.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:39.878+0000 7f46e8c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:37:39.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:39.879+0000 7f46e8c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T11:37:42.719 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T11:37:42.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:42.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:42.850+0000 7f2031b1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T11:37:42.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:42.850+0000 7f2031b1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T11:37:42.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:42.850+0000 7f2031b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:37:42.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:42.850+0000 7f2031b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:37:42.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:42.850+0000 7f2031b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:37:42.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:37:42.851+0000 7f2031b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T11:37:45.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T11:37:45.718 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T11:37:45.747 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:45.747 INFO:tasks.ceph:role: osd.4 2024-04-25T11:37:45.747 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:37:45.747 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T11:37:45.823 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:37:45.824 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:37:45.824 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:37:45.824 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:37:45.824 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:37:45.824 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:37:45.824 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:37:45.824 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:37:45.824 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:37:45.824 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:37:46.330 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-25T11:37:46.332 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-25T11:37:46.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T11:37:46.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T11:37:46.475 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T11:37:46.540 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:46.540 INFO:tasks.ceph:role: osd.5 2024-04-25T11:37:46.540 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:37:46.540 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:37:46.612 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:37:47.083 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-25T11:37:47.085 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-25T11:37:47.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T11:37:47.179 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T11:37:47.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T11:37:47.272 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:47.272 INFO:tasks.ceph:role: osd.6 2024-04-25T11:37:47.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:37:47.272 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:37:47.344 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:37:47.833 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-25T11:37:47.835 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-25T11:37:47.835 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T11:37:47.946 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T11:37:47.976 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T11:37:48.042 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:37:48.042 INFO:tasks.ceph:role: osd.7 2024-04-25T11:37:48.042 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:37:48.042 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T11:37:48.114 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:37:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:37:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:37:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:37:48.115 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:37:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:37:48.115 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:37:48.115 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:37:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:37:48.115 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:37:48.616 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-25T11:37:48.618 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-25T11:37:48.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T11:37:48.764 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T11:37:48.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:48.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:48.903+0000 7fc2c1926640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T11:37:48.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:48.904+0000 7fc2c1926640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T11:37:48.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:48.904+0000 7fc2c1926640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:37:48.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:48.904+0000 7fc2c1926640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:37:48.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:48.904+0000 7fc2c1926640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:37:48.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:48.904+0000 7fc2c1926640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T11:37:51.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T11:37:51.773 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:51.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:51.877+0000 7f38331dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T11:37:51.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:51.877+0000 7f38331dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T11:37:51.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:51.877+0000 7f38331dc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:37:51.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:51.877+0000 7f38331dc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:37:51.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:51.877+0000 7f38331dc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:37:51.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:51.878+0000 7f38331dc640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T11:37:54.717 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T11:37:54.746 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:54.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:54.849+0000 7fc509cfd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T11:37:54.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:54.849+0000 7fc509cfd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T11:37:54.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:54.849+0000 7fc509cfd640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:37:54.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:54.849+0000 7fc509cfd640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:37:54.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:54.849+0000 7fc509cfd640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:37:54.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:54.850+0000 7fc509cfd640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T11:37:57.563 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:57.566 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:57.689 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T11:37:57.715 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:37:57.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:57.814+0000 7f31af6d9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T11:37:57.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:57.814+0000 7f31af6d9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T11:37:57.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:57.814+0000 7f31af6d9640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:37:57.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:57.814+0000 7f31af6d9640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:37:57.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:57.814+0000 7f31af6d9640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:37:57.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:37:57.815+0000 7f31af6d9640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T11:38:00.649 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T11:38:00.678 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T11:38:00.678 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:00.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T11:38:00.708 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:00.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T11:38:00.777 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:00.777 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T11:38:00.844 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:00.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-25T11:38:00.910 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:00.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T11:38:00.977 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:00.977 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T11:38:01.042 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:01.042 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T11:38:01.107 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:01.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T11:38:01.171 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:01.171 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T11:38:01.201 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:01.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T11:38:01.269 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:01.269 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T11:38:01.336 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:01.336 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-25T11:38:01.402 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:01.403 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T11:38:01.469 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:01.469 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T11:38:01.535 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:01.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T11:38:01.600 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:01.600 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T11:38:01.666 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:01.666 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T11:38:01.684 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T11:38:01.684 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T11:38:01.726 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T11:38:01.750 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2024-04-25T11:38:01.750 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6QCpmBx8ZABAAOs97D5jZXbZ7fFxoO8FIfQ== 2024-04-25T11:38:01.750 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-04-25T11:38:01.751 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6QCpmaXksCBAA1PIsQFFBK7wDUcBEVofyhQ== 2024-04-25T11:38:01.751 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-04-25T11:38:01.751 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6QCpmapoZDxAA13cpGUM90pyeApWwnOn2EQ== 2024-04-25T11:38:01.751 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.e] 2024-04-25T11:38:01.751 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6QCpmDegOFhAAH92hdAtUggnDaLJzhC+B7Q== 2024-04-25T11:38:01.751 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2024-04-25T11:38:01.751 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB9QCpmsuYlNxAAcacBvfCET9cHobvAVdLQSA== 2024-04-25T11:38:01.751 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCAQCpmeaQJNhAA6MPx65ACBd/Y4MPLlUX4uQ== 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCDQCpmI/xoNBAAD4crCGA1Z16OyRAIb/XjPg== 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCGQCpmmvq4MhAA5ix2UStohIScQ9AzDnFmmg== 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6QCpmMW4HBRAAGzTMRhU8/G4886njUBj0Nw== 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6QCpm6enPGhAAzBlHm7thLnAMa9sBb4dtJQ== 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-04-25T11:38:01.752 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6QCpmfG7NIRAA/Tvjl9ukbfzrtyjkRAGGDg== 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.f] 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6QCpmaGSrKBAAisml4fcgUasktsaeVqqu3Q== 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCMQCpmgJr8NRAA5qv3c9y4DTbCDx3A4XJrBg== 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCPQCpmMyhlNBAAYPGRwggKpJLrxKulWcmo0g== 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCSQCpms4ytMhAA5PYP/n1Rv+Gb+ZlamH+0hg== 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQCVQCpmRrGZMBAAE1HtFXSDwwlU+IaXeMCxLA== 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T11:38:01.753 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQB6QCpmI+yvLBAAtB+aDvCqptoX+OAIOFRnpw== 2024-04-25T11:38:01.755 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2024-04-25T11:38:01.755 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6QCpmBx8ZABAAOs97D5jZXbZ7fFxoO8FIfQ== 2024-04-25T11:38:01.755 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-04-25T11:38:01.755 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6QCpmaXksCBAA1PIsQFFBK7wDUcBEVofyhQ== 2024-04-25T11:38:01.755 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-04-25T11:38:01.755 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6QCpmapoZDxAA13cpGUM90pyeApWwnOn2EQ== 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.e] 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6QCpmDegOFhAAH92hdAtUggnDaLJzhC+B7Q== 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB9QCpmsuYlNxAAcacBvfCET9cHobvAVdLQSA== 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCAQCpmeaQJNhAA6MPx65ACBd/Y4MPLlUX4uQ== 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCDQCpmI/xoNBAAD4crCGA1Z16OyRAIb/XjPg== 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCGQCpmmvq4MhAA5ix2UStohIScQ9AzDnFmmg== 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6QCpmMW4HBRAAGzTMRhU8/G4886njUBj0Nw== 2024-04-25T11:38:01.756 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6QCpm6enPGhAAzBlHm7thLnAMa9sBb4dtJQ== 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6QCpmfG7NIRAA/Tvjl9ukbfzrtyjkRAGGDg== 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.f] 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6QCpmaGSrKBAAisml4fcgUasktsaeVqqu3Q== 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCMQCpmgJr8NRAA5qv3c9y4DTbCDx3A4XJrBg== 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCPQCpmMyhlNBAAYPGRwggKpJLrxKulWcmo0g== 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCSQCpms4ytMhAA5PYP/n1Rv+Gb+ZlamH+0hg== 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCVQCpmRrGZMBAAE1HtFXSDwwlU+IaXeMCxLA== 2024-04-25T11:38:01.757 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-25T11:38:01.758 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB6QCpmI+yvLBAAtB+aDvCqptoX+OAIOFRnpw== 2024-04-25T11:38:01.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:38:01.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:38:01.850 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:01.852 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:01.905 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:01.908 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:01.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:01.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:02.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.020 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.069 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.125 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.176 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.181 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.232 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:38:02.234 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:38:02.284 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:02.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:02.339 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:02.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:02.394 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:02.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:38:02.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.451 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.501 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.503 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.605 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:38:02.658 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:38:02.660 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:38:02.713 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T11:38:02.713 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T11:38:02.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:38:02.833 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T11:38:02.857 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T11:38:02.881 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:38:02.976 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T11:38:03.000 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T11:38:03.064 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:38:03.159 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T11:38:03.183 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:38:03.185 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:38:03.238 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T11:38:03.238 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T11:38:03.238 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T11:38:03.241 INFO:tasks.ceph.mon.a:Started 2024-04-25T11:38:03.241 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T11:38:03.241 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T11:38:03.280 INFO:tasks.ceph.mon.b:Started 2024-04-25T11:38:03.281 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T11:38:03.281 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T11:38:03.283 INFO:tasks.ceph.mon.c:Started 2024-04-25T11:38:03.283 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T11:38:03.283 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T11:38:03.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T11:38:03.287 INFO:tasks.ceph.mgr.y:Started 2024-04-25T11:38:03.287 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T11:38:03.287 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T11:38:03.289 INFO:tasks.ceph.mgr.x:Started 2024-04-25T11:38:03.289 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T11:38:03.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T11:38:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.402+0000 7efe05c9a640 1 Processor -- start 2024-04-25T11:38:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.402+0000 7efe05c9a640 1 -- start start 2024-04-25T11:38:03.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.402+0000 7efe05c9a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:03.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.402+0000 7efe05c9a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:03.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.402+0000 7efe05c9a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.402+0000 7efe05c9a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efe000c64d0 con 0x7efe00151bc0 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.402+0000 7efe05c9a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7efe000c8a80 con 0x7efe00151fa0 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.402+0000 7efe05c9a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efe000c8d60 con 0x7efe000c4f30 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.402+0000 7efdff7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdff7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50094/0 (socket says 172.21.15.40:50094) 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdff7fe640 1 -- 172.21.15.40:0/2444693283 learned_addr learned my addr 172.21.15.40:0/2444693283 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdff7fe640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 msgr2=0x7efe000c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdff7fe640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 msgr2=0x7efe000c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 msgr2=0x7efe000c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 msgr2=0x7efe000c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:38:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:38:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfffff640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe000c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T11:38:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfffff640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe000c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.403+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:38:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdff7fe640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 msgr2=0x7efe000c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:38:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdff7fe640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 msgr2=0x7efe000c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:38:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 msgr2=0x7efe000c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:38:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 msgr2=0x7efe000c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfffff640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe000c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T11:38:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfffff640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe000c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:38:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:03.604+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdff7fe640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 msgr2=0x7efe000c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdff7fe640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 msgr2=0x7efe000c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 msgr2=0x7efe000c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfffff640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe000c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:38:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 msgr2=0x7efe000c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfffff640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe000c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:38:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.005+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:38:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdff7fe640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 msgr2=0x7efe000c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:38:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdff7fe640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 msgr2=0x7efe000c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfffff640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe000c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 msgr2=0x7efe000c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfffff640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe000c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 msgr2=0x7efe000c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:38:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:04.806+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:38:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:06.403+0000 7efdfd7fa640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe000c7050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:38:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:06.403+0000 7efdfd7fa640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe000c7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:06.403+0000 7efdfd7fa640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 msgr2=0x7efe000c49f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:38:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:06.403+0000 7efdfd7fa640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.62:3300/0 conn(0x7efe00151fa0 0x7efe000c49f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:06.403+0000 7efdfd7fa640 1 -- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 msgr2=0x7efe000c8340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:38:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:06.403+0000 7efdfd7fa640 1 --2- 172.21.15.40:0/2444693283 >> v2:172.21.15.40:3300/0 conn(0x7efe00151bc0 0x7efe000c8340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:06.403+0000 7efdfd7fa640 1 -- 172.21.15.40:0/2444693283 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe000c9040 con 0x7efde8007320 2024-04-25T11:38:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:06.403+0000 7efdfd7fa640 1 -- 172.21.15.40:0/2444693283 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde800c070 con 0x7efde80036b0 2024-04-25T11:38:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.570+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2746431514 0 0) 0x7efe000c9040 con 0x7efde8007320 2024-04-25T11:38:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.570+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdec003520 con 0x7efde8007320 2024-04-25T11:38:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4190000295 0 0) 0x7efde800c070 con 0x7efde80036b0 2024-04-25T11:38:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe000c9040 con 0x7efde80036b0 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2941102555 0 0) 0x7efe000c9040 con 0x7efde80036b0 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 >> v1:172.21.15.40:6789/0 conn(0x7efde8007320 legacy=0x7efde80097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde800d310 con 0x7efde80036b0 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7efdf000e780 con 0x7efde80036b0 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 >> v1:172.21.15.62:6789/0 conn(0x7efde80036b0 legacy=0x7efde8005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efdec003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfffff640 1 -- 172.21.15.40:0/2444693283 reap_dead start 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004170 0x7efdec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004f40 0x7efdec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdec00bf00 con 0x7efdec004170 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efdec00c080 con 0x7efdec004f40 2024-04-25T11:38:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efdec00c200 con 0x7efe000c4f30 2024-04-25T11:38:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004170 0x7efdec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdff7fe640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efdec003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.571+0000 7efdfffff640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004f40 0x7efdec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.572+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efdec003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.572+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efdec003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.572+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004f40 msgr2=0x7efdec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.572+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004f40 0x7efdec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.572+0000 7efdfeffd640 1 -- 172.21.15.40:0/2444693283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe000c9040 con 0x7efdec004170 2024-04-25T11:38:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.573+0000 7efdfeffd640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004170 0x7efdec004800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe00158c20 tx=0x7efdec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a20a9f2a482288 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.574+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4031070 con 0x7efdec004170 2024-04-25T11:38:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.574+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf402b860 con 0x7efdec004170 2024-04-25T11:38:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.574+0000 7efdfe7fc640 1 -- 172.21.15.40:0/2444693283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4011890 con 0x7efdec004170 2024-04-25T11:38:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.574+0000 7efe05c9a640 1 -- 172.21.15.40:0/2444693283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004170 msgr2=0x7efdec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.574+0000 7efe05c9a640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004170 0x7efdec004800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe00158c20 tx=0x7efdec005830 comp rx=0 tx=0).stop 2024-04-25T11:38:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.575+0000 7efe05c9a640 1 -- 172.21.15.40:0/2444693283 shutdown_connections 2024-04-25T11:38:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.575+0000 7efe05c9a640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004f40 0x7efdec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.575+0000 7efe05c9a640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004170 0x7efdec004800 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.575+0000 7efe05c9a640 1 --2- 172.21.15.40:0/2444693283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efdec003be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.575+0000 7efe05c9a640 1 -- 172.21.15.40:0/2444693283 >> 172.21.15.40:0/2444693283 conn(0x7efe000703d0 msgr2=0x7efe000707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.575+0000 7efe05c9a640 1 -- 172.21.15.40:0/2444693283 shutdown_connections 2024-04-25T11:38:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.575+0000 7efe05c9a640 1 -- 172.21.15.40:0/2444693283 wait complete. 2024-04-25T11:38:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.576+0000 7efe05c9a640 1 Processor -- start 2024-04-25T11:38:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efe05c9a640 1 -- start start 2024-04-25T11:38:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efe05c9a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004170 0x7efe001536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efe05c9a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004f40 0x7efe00154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efe05c9a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe00154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efe05c9a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efe000c9320 con 0x7efdec004f40 2024-04-25T11:38:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efe05c9a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efe000c6b30 con 0x7efdec004170 2024-04-25T11:38:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efe05c9a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efe000c49f0 con 0x7efe000c4f30 2024-04-25T11:38:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efdfeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004f40 0x7efe00154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efdfeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004f40 0x7efe00154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50172/0 (socket says 172.21.15.40:50172) 2024-04-25T11:38:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efdfeffd640 1 -- 172.21.15.40:0/409684131 learned_addr learned my addr 172.21.15.40:0/409684131 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efdfffff640 1 --2- 172.21.15.40:0/409684131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe00154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.577+0000 7efdff7fe640 1 --2- 172.21.15.40:0/409684131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004170 0x7efe001536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efdfeffd640 1 -- 172.21.15.40:0/409684131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 msgr2=0x7efe00154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efdfeffd640 1 --2- 172.21.15.40:0/409684131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe00154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efdfeffd640 1 -- 172.21.15.40:0/409684131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004170 msgr2=0x7efe001536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efdfeffd640 1 --2- 172.21.15.40:0/409684131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004170 0x7efe001536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efdfeffd640 1 -- 172.21.15.40:0/409684131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe001ec620 con 0x7efdec004f40 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efdfeffd640 1 --2- 172.21.15.40:0/409684131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004f40 0x7efe00154900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdf4011d50 tx=0x7efdf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efde7fff640 1 -- 172.21.15.40:0/409684131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4042020 con 0x7efdec004f40 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efde7fff640 1 -- 172.21.15.40:0/409684131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf4035630 con 0x7efdec004f40 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efe05c9a640 1 -- 172.21.15.40:0/409684131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe00152380 con 0x7efdec004f40 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efde7fff640 1 -- 172.21.15.40:0/409684131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf403a040 con 0x7efdec004f40 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.578+0000 7efe05c9a640 1 -- 172.21.15.40:0/409684131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe00152920 con 0x7efdec004f40 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.579+0000 7efde7fff640 1 -- 172.21.15.40:0/409684131 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efdf4045020 con 0x7efdec004f40 2024-04-25T11:38:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.579+0000 7efde7fff640 1 -- 172.21.15.40:0/409684131 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efdf4047c20 con 0x7efdec004f40 2024-04-25T11:38:08.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.580+0000 7efe05c9a640 1 -- 172.21.15.40:0/409684131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe00069200 con 0x7efdec004f40 2024-04-25T11:38:08.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.582+0000 7efde7fff640 1 -- 172.21.15.40:0/409684131 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efdf404c020 con 0x7efdec004f40 2024-04-25T11:38:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.651+0000 7efe05c9a640 1 -- 172.21.15.40:0/409684131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7efe000caa20 con 0x7efdec004f40 2024-04-25T11:38:08.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.654+0000 7efde7fff640 1 -- 172.21.15.40:0/409684131 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7efdf4011890 con 0x7efdec004f40 2024-04-25T11:38:08.642 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-04-25T11:38:08.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.655+0000 7efe05c9a640 1 -- 172.21.15.40:0/409684131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004f40 msgr2=0x7efe00154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.655+0000 7efe05c9a640 1 --2- 172.21.15.40:0/409684131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004f40 0x7efe00154900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdf4011d50 tx=0x7efdf40046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.655+0000 7efe05c9a640 1 -- 172.21.15.40:0/409684131 shutdown_connections 2024-04-25T11:38:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.655+0000 7efe05c9a640 1 --2- 172.21.15.40:0/409684131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe000c4f30 0x7efe00154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.655+0000 7efe05c9a640 1 --2- 172.21.15.40:0/409684131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdec004f40 0x7efe00154900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.655+0000 7efe05c9a640 1 --2- 172.21.15.40:0/409684131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec004170 0x7efe001536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.655+0000 7efe05c9a640 1 -- 172.21.15.40:0/409684131 >> 172.21.15.40:0/409684131 conn(0x7efe000703d0 msgr2=0x7efe0014d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.655+0000 7efe05c9a640 1 -- 172.21.15.40:0/409684131 shutdown_connections 2024-04-25T11:38:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:08.655+0000 7efe05c9a640 1 -- 172.21.15.40:0/409684131 wait complete. 2024-04-25T11:38:08.654 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T11:38:08.654 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T11:38:08.654 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:08.654 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T11:38:08.679 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:08.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T11:38:08.682 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:08.695+0000 7f5bbbf21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:38:08.744 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:08.745 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T11:38:08.809 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:38:08.810 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T11:38:08.842 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:08.855+0000 7f5bbbf21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:38:08.874 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:08.874 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T11:38:08.898 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:08.911+0000 7f5bbbf21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:38:08.899 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:08.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T11:38:08.964 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:08.964 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T11:38:09.028 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T11:38:09.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T11:38:09.093 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new f522bed8-ab40-43ae-8616-996ef5e1732c 0 2024-04-25T11:38:09.172 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:09.185+0000 7f5bbbf21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:38:09.221 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:09.233+0000 7f5bbbf21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:38:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.247+0000 7f0049543640 1 Processor -- start 2024-04-25T11:38:09.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.247+0000 7f0049543640 1 -- start start 2024-04-25T11:38:09.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.247+0000 7f0049543640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f004415b8e0 0x7f00440d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.247+0000 7f0049543640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00440cbe40 con 0x7f00440cc820 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.247+0000 7f0049543640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00440cb880 con 0x7f0044155070 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f0049543640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f004415be20 con 0x7f004415b8e0 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f0042ffd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0044155070 0x7f004415ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:46406/0 (socket says 172.21.15.62:46406) 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f0042ffd640 1 -- 172.21.15.62:0/3094601389 learned_addr learned my addr 172.21.15.62:0/3094601389 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f00427fc640 1 --2- 172.21.15.62:0/3094601389 >> v2:172.21.15.40:3300/0 conn(0x7f004415b8e0 0x7f00440d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f0041ffb640 1 -- 172.21.15.62:0/3094601389 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 177976960 0 0) 0x7f00440cb880 con 0x7f0044155070 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f0041ffb640 1 -- 172.21.15.62:0/3094601389 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f002c003520 con 0x7f0044155070 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f0041ffb640 1 -- 172.21.15.62:0/3094601389 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397136196 0 0) 0x7f00440cbe40 con 0x7f00440cc820 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f0041ffb640 1 -- 172.21.15.62:0/3094601389 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00440cb880 con 0x7f00440cc820 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f0041ffb640 1 -- 172.21.15.62:0/3094601389 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 209398583 0 0) 0x7f002c003520 con 0x7f0044155070 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f00427fc640 1 -- 172.21.15.62:0/3094601389 >> v1:172.21.15.62:6789/0 conn(0x7f0044155070 legacy=0x7f004415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f00427fc640 1 -- 172.21.15.62:0/3094601389 >> v1:172.21.15.40:6789/0 conn(0x7f00440cc820 legacy=0x7f00441657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.248+0000 7f00427fc640 1 -- 172.21.15.62:0/3094601389 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00440cbb60 con 0x7f004415b8e0 2024-04-25T11:38:09.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.249+0000 7f00427fc640 1 --2- 172.21.15.62:0/3094601389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004415b8e0 0x7f00440d04f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0038009720 tx=0x7f003802a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cde55fa1e4d214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.249+0000 7f0041ffb640 1 -- 172.21.15.62:0/3094601389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038037070 con 0x7f004415b8e0 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.249+0000 7f0041ffb640 1 -- 172.21.15.62:0/3094601389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003802abe0 con 0x7f004415b8e0 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.249+0000 7f0041ffb640 1 -- 172.21.15.62:0/3094601389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034001670 con 0x7f004415b8e0 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.250+0000 7f0049543640 1 -- 172.21.15.62:0/3094601389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004415b8e0 msgr2=0x7f00440d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.250+0000 7f0049543640 1 --2- 172.21.15.62:0/3094601389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004415b8e0 0x7f00440d04f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0038009720 tx=0x7f003802a0d0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.250+0000 7f0049543640 1 -- 172.21.15.62:0/3094601389 shutdown_connections 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.250+0000 7f0049543640 1 --2- 172.21.15.62:0/3094601389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004415b8e0 0x7f00440d04f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.250+0000 7f0049543640 1 -- 172.21.15.62:0/3094601389 >> 172.21.15.62:0/3094601389 conn(0x7f004406c680 msgr2=0x7f004406e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.250+0000 7f0049543640 1 -- 172.21.15.62:0/3094601389 shutdown_connections 2024-04-25T11:38:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.250+0000 7f0049543640 1 -- 172.21.15.62:0/3094601389 wait complete. 2024-04-25T11:38:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.251+0000 7f0049543640 1 Processor -- start 2024-04-25T11:38:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.251+0000 7f0049543640 1 -- start start 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0049543640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00440cc820 0x7f00440c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0049543640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0044155070 0x7f00440c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0049543640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f004415b8e0 0x7f0044159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0049543640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f00440cb4e0 con 0x7f0044155070 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0049543640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f004415e840 con 0x7f00440cc820 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0049543640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0044155450 con 0x7f004415b8e0 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0042ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00440cc820 0x7f00440c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f00437fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f004415b8e0 0x7f0044159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0042ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00440cc820 0x7f00440c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41456/0 (socket says 172.21.15.62:41456) 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0042ffd640 1 -- 172.21.15.62:0/3244541131 learned_addr learned my addr 172.21.15.62:0/3244541131 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f00427fc640 1 --2- 172.21.15.62:0/3244541131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0044155070 0x7f00440c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0042ffd640 1 -- 172.21.15.62:0/3244541131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f004415b8e0 msgr2=0x7f0044159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0042ffd640 1 --2- 172.21.15.62:0/3244541131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f004415b8e0 0x7f0044159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0042ffd640 1 -- 172.21.15.62:0/3244541131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0044155070 msgr2=0x7f00440c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0042ffd640 1 --2- 172.21.15.62:0/3244541131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0044155070 0x7f00440c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0042ffd640 1 -- 172.21.15.62:0/3244541131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00440c71a0 con 0x7f00440cc820 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f00437fe640 1 --2- 172.21.15.62:0/3244541131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f004415b8e0 0x7f0044159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.252+0000 7f0042ffd640 1 --2- 172.21.15.62:0/3244541131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00440cc820 0x7f00440c5ee0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f003400c750 tx=0x7f003400cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.253+0000 7f002b7fe640 1 -- 172.21.15.62:0/3244541131 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034018070 con 0x7f00440cc820 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.253+0000 7f002b7fe640 1 -- 172.21.15.62:0/3244541131 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00340072d0 con 0x7f00440cc820 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.253+0000 7f0049543640 1 -- 172.21.15.62:0/3244541131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00441f0ba0 con 0x7f00440cc820 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.253+0000 7f002b7fe640 1 -- 172.21.15.62:0/3244541131 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00340047e0 con 0x7f00440cc820 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.253+0000 7f0049543640 1 -- 172.21.15.62:0/3244541131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441f1140 con 0x7f00440cc820 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.253+0000 7f002b7fe640 1 -- 172.21.15.62:0/3244541131 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f003401b070 con 0x7f00440cc820 2024-04-25T11:38:09.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.253+0000 7f002b7fe640 1 -- 172.21.15.62:0/3244541131 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f003401d440 con 0x7f00440cc820 2024-04-25T11:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.254+0000 7f0049543640 1 -- 172.21.15.62:0/3244541131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0044068bb0 con 0x7f00440cc820 2024-04-25T11:38:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.256+0000 7f002b7fe640 1 -- 172.21.15.62:0/3244541131 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f003400ae20 con 0x7f00440cc820 2024-04-25T11:38:09.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.330+0000 7f0049543640 1 -- 172.21.15.62:0/3244541131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f522bed8-ab40-43ae-8616-996ef5e1732c", "id": 0} v 0) -- 0x7f00440c3e40 con 0x7f00440cc820 2024-04-25T11:38:09.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.334+0000 7f002b7fe640 1 -- 172.21.15.62:0/3244541131 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f522bed8-ab40-43ae-8616-996ef5e1732c", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f003400a640 con 0x7f00440cc820 2024-04-25T11:38:09.321 INFO:teuthology.orchestra.run.smithi062.stdout:0 2024-04-25T11:38:09.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.335+0000 7f0049543640 1 -- 172.21.15.62:0/3244541131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00440cc820 msgr2=0x7f00440c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.335+0000 7f0049543640 1 --2- 172.21.15.62:0/3244541131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00440cc820 0x7f00440c5ee0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f003400c750 tx=0x7f003400cc10 comp rx=0 tx=0).stop 2024-04-25T11:38:09.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.335+0000 7f0049543640 1 -- 172.21.15.62:0/3244541131 shutdown_connections 2024-04-25T11:38:09.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.335+0000 7f0049543640 1 --2- 172.21.15.62:0/3244541131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f004415b8e0 0x7f0044159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.335+0000 7f0049543640 1 --2- 172.21.15.62:0/3244541131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0044155070 0x7f00440c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.335+0000 7f0049543640 1 --2- 172.21.15.62:0/3244541131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00440cc820 0x7f00440c5ee0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.335+0000 7f0049543640 1 -- 172.21.15.62:0/3244541131 >> 172.21.15.62:0/3244541131 conn(0x7f004406c680 msgr2=0x7f0044158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.335+0000 7f0049543640 1 -- 172.21.15.62:0/3244541131 shutdown_connections 2024-04-25T11:38:09.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.335+0000 7f0049543640 1 -- 172.21.15.62:0/3244541131 wait complete. 2024-04-25T11:38:09.333 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 713a8175-952b-4888-90e2-9da6f79016a3 1 2024-04-25T11:38:09.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.486+0000 7fcf6e0ae640 1 Processor -- start 2024-04-25T11:38:09.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.486+0000 7fcf6e0ae640 1 -- start start 2024-04-25T11:38:09.474 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:09.486+0000 7feccc419040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:38:09.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf6e0ae640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf68071a60 0x7fcf680c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf6e0ae640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcf6815b960 0x7fcf68154990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf6e0ae640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf680c9630 con 0x7fcf68071e40 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf6e0ae640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcf6806d3a0 con 0x7fcf68071a60 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf6e0ae640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fcf6806d680 con 0x7fcf6815b960 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf67fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcf6815b960 0x7fcf68154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf67fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcf6815b960 0x7fcf68154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41470/0 (socket says 172.21.15.62:41470) 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf67fff640 1 -- 172.21.15.62:0/3047671183 learned_addr learned my addr 172.21.15.62:0/3047671183 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf677fe640 1 --2- 172.21.15.62:0/3047671183 >> v2:172.21.15.40:3300/0 conn(0x7fcf68071a60 0x7fcf680c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf67fff640 1 -- 172.21.15.62:0/3047671183 >> v2:172.21.15.40:3300/0 conn(0x7fcf68071a60 msgr2=0x7fcf680c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf67fff640 1 --2- 172.21.15.62:0/3047671183 >> v2:172.21.15.40:3300/0 conn(0x7fcf68071a60 0x7fcf680c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf67fff640 1 -- 172.21.15.62:0/3047671183 >> v1:172.21.15.40:6789/0 conn(0x7fcf68071e40 legacy=0x7fcf6815b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf67fff640 1 -- 172.21.15.62:0/3047671183 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6806d960 con 0x7fcf6815b960 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.487+0000 7fcf667fc640 1 -- 172.21.15.62:0/3047671183 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4181233894 0 0) 0x7fcf680c9630 con 0x7fcf68071e40 2024-04-25T11:38:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.488+0000 7fcf67fff640 1 --2- 172.21.15.62:0/3047671183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf6815b960 0x7fcf68154990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806d290 tx=0x7fcf6002a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7a7a95ba224764b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.488+0000 7fcf667fc640 1 -- 172.21.15.62:0/3047671183 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60036070 con 0x7fcf6815b960 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.488+0000 7fcf667fc640 1 -- 172.21.15.62:0/3047671183 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf60031600 con 0x7fcf6815b960 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.488+0000 7fcf667fc640 1 -- 172.21.15.62:0/3047671183 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600319f0 con 0x7fcf6815b960 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.489+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/3047671183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf6815b960 msgr2=0x7fcf68154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.489+0000 7fcf6e0ae640 1 --2- 172.21.15.62:0/3047671183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf6815b960 0x7fcf68154990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806d290 tx=0x7fcf6002a7c0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.489+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/3047671183 shutdown_connections 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.489+0000 7fcf6e0ae640 1 --2- 172.21.15.62:0/3047671183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf6815b960 0x7fcf68154990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.489+0000 7fcf6e0ae640 1 --2- 172.21.15.62:0/3047671183 >> v2:172.21.15.40:3300/0 conn(0x7fcf68071a60 0x7fcf680c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.489+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/3047671183 >> 172.21.15.62:0/3047671183 conn(0x7fcf6806b970 msgr2=0x7fcf6806bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.489+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/3047671183 shutdown_connections 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.489+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/3047671183 wait complete. 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.490+0000 7fcf6e0ae640 1 Processor -- start 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.490+0000 7fcf6e0ae640 1 -- start start 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf6e0ae640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf68071a60 0x7fcf68155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf6e0ae640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf68071e40 0x7fcf681e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf6e0ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6815b960 0x7fcf681f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf6e0ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680cab90 con 0x7fcf6815b960 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf6e0ae640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6806df20 con 0x7fcf68071a60 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf6e0ae640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcf680cad10 con 0x7fcf68071e40 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf68071a60 0x7fcf68155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf68071a60 0x7fcf68155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41472/0 (socket says 172.21.15.62:41472) 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf66ffd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf68071e40 0x7fcf681e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf677fe640 1 -- 172.21.15.62:0/183712767 learned_addr learned my addr 172.21.15.62:0/183712767 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf67fff640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6815b960 0x7fcf681f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf677fe640 1 -- 172.21.15.62:0/183712767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf68071e40 msgr2=0x7fcf681e25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf677fe640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf68071e40 0x7fcf681e25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf677fe640 1 -- 172.21.15.62:0/183712767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6815b960 msgr2=0x7fcf681f09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf677fe640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6815b960 0x7fcf681f09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf677fe640 1 -- 172.21.15.62:0/183712767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf68153c20 con 0x7fcf68071a60 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf66ffd640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf68071e40 0x7fcf681e25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.491+0000 7fcf67fff640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6815b960 0x7fcf681f09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:09.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.492+0000 7fcf677fe640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf68071a60 0x7fcf68155330 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e9b0 tx=0x7fcf5800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.492+0000 7fcf5ffff640 1 -- 172.21.15.62:0/183712767 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58017070 con 0x7fcf68071a60 2024-04-25T11:38:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.492+0000 7fcf5ffff640 1 -- 172.21.15.62:0/183712767 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5800c9a0 con 0x7fcf68071a60 2024-04-25T11:38:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.492+0000 7fcf5ffff640 1 -- 172.21.15.62:0/183712767 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800cd90 con 0x7fcf68071a60 2024-04-25T11:38:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.492+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/183712767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf68153f00 con 0x7fcf68071a60 2024-04-25T11:38:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.492+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/183712767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf681f0490 con 0x7fcf68071a60 2024-04-25T11:38:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.492+0000 7fcf5ffff640 1 -- 172.21.15.62:0/183712767 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf5801a070 con 0x7fcf68071a60 2024-04-25T11:38:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.492+0000 7fcf5ffff640 1 -- 172.21.15.62:0/183712767 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fcf58013820 con 0x7fcf68071a60 2024-04-25T11:38:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.493+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/183712767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf68069200 con 0x7fcf68071a60 2024-04-25T11:38:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.497+0000 7fcf5ffff640 1 -- 172.21.15.62:0/183712767 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcf580066b0 con 0x7fcf68071a60 2024-04-25T11:38:09.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.570+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/183712767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "713a8175-952b-4888-90e2-9da6f79016a3", "id": 1} v 0) -- 0x7fcf680cb410 con 0x7fcf68071a60 2024-04-25T11:38:09.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.576+0000 7fcf5ffff640 1 -- 172.21.15.62:0/183712767 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "713a8175-952b-4888-90e2-9da6f79016a3", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fcf58006020 con 0x7fcf68071a60 2024-04-25T11:38:09.564 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T11:38:09.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.577+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/183712767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf68071a60 msgr2=0x7fcf68155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.577+0000 7fcf6e0ae640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf68071a60 0x7fcf68155330 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e9b0 tx=0x7fcf5800ee70 comp rx=0 tx=0).stop 2024-04-25T11:38:09.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.577+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/183712767 shutdown_connections 2024-04-25T11:38:09.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.577+0000 7fcf6e0ae640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6815b960 0x7fcf681f09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.577+0000 7fcf6e0ae640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf68071e40 0x7fcf681e25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.577+0000 7fcf6e0ae640 1 --2- 172.21.15.62:0/183712767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf68071a60 0x7fcf68155330 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.577+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/183712767 >> 172.21.15.62:0/183712767 conn(0x7fcf6806b970 msgr2=0x7fcf68158440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.577+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/183712767 shutdown_connections 2024-04-25T11:38:09.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.577+0000 7fcf6e0ae640 1 -- 172.21.15.62:0/183712767 wait complete. 2024-04-25T11:38:09.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 789f733e-f414-4093-9ad1-444b395333a5 2 2024-04-25T11:38:09.632 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:09.645+0000 7feccc419040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:38:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.689+0000 7fd5d4b5d640 1 Processor -- start 2024-04-25T11:38:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.689+0000 7fd5d4b5d640 1 -- start start 2024-04-25T11:38:09.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5d4b5d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d0155070 0x7fd5d015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5d4b5d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd5d015b8e0 0x7fd5d00d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5d4b5d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5d00cc820 0x7fd5d00d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5d4b5d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d015be20 con 0x7fd5d00cc820 2024-04-25T11:38:09.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5d4b5d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d00cbe40 con 0x7fd5d015b8e0 2024-04-25T11:38:09.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5d4b5d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d00cb880 con 0x7fd5d0155070 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5ce59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d0155070 0x7fd5d015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5cdd9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd5d015b8e0 0x7fd5d00d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5cdd9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd5d015b8e0 0x7fd5d00d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41482/0 (socket says 172.21.15.62:41482) 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5ce59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d0155070 0x7fd5d015ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:47102/0 (socket says 172.21.15.62:47102) 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5cdd9b640 1 -- 172.21.15.62:0/2754797301 learned_addr learned my addr 172.21.15.62:0/2754797301 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5ced9d640 1 --2- 172.21.15.62:0/2754797301 >> v2:172.21.15.40:3300/0 conn(0x7fd5d00cc820 0x7fd5d00d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5cdd9b640 1 -- 172.21.15.62:0/2754797301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d0155070 msgr2=0x7fd5d015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5cdd9b640 1 --2- 172.21.15.62:0/2754797301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d0155070 0x7fd5d015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5cdd9b640 1 -- 172.21.15.62:0/2754797301 >> v2:172.21.15.40:3300/0 conn(0x7fd5d00cc820 msgr2=0x7fd5d00d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5cdd9b640 1 --2- 172.21.15.62:0/2754797301 >> v2:172.21.15.40:3300/0 conn(0x7fd5d00cc820 0x7fd5d00d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.690+0000 7fd5cdd9b640 1 -- 172.21.15.62:0/2754797301 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d00cbb60 con 0x7fd5d015b8e0 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.691+0000 7fd5cdd9b640 1 --2- 172.21.15.62:0/2754797301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5d015b8e0 0x7fd5d00d04f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd5d006cc70 tx=0x7fd5c402a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bd4721f21d02e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.691+0000 7fd5cd59a640 1 -- 172.21.15.62:0/2754797301 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4037070 con 0x7fd5d015b8e0 2024-04-25T11:38:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.691+0000 7fd5cd59a640 1 -- 172.21.15.62:0/2754797301 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c4032440 con 0x7fd5d015b8e0 2024-04-25T11:38:09.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.691+0000 7fd5cd59a640 1 -- 172.21.15.62:0/2754797301 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4032810 con 0x7fd5d015b8e0 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.691+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/2754797301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5d015b8e0 msgr2=0x7fd5d00d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.691+0000 7fd5d4b5d640 1 --2- 172.21.15.62:0/2754797301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5d015b8e0 0x7fd5d00d04f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd5d006cc70 tx=0x7fd5c402a140 comp rx=0 tx=0).stop 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.692+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/2754797301 shutdown_connections 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.692+0000 7fd5d4b5d640 1 --2- 172.21.15.62:0/2754797301 >> v2:172.21.15.40:3300/0 conn(0x7fd5d00cc820 0x7fd5d00d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.692+0000 7fd5d4b5d640 1 --2- 172.21.15.62:0/2754797301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5d015b8e0 0x7fd5d00d04f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.692+0000 7fd5d4b5d640 1 --2- 172.21.15.62:0/2754797301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d0155070 0x7fd5d015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.692+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/2754797301 >> 172.21.15.62:0/2754797301 conn(0x7fd5d006c680 msgr2=0x7fd5d006e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.692+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/2754797301 shutdown_connections 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.692+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/2754797301 wait complete. 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.693+0000 7fd5d4b5d640 1 Processor -- start 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.693+0000 7fd5d4b5d640 1 -- start start 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5d4b5d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d00cc820 0x7fd5d01f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5d4b5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0155070 0x7fd5d00c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5d4b5d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5d015b8e0 0x7fd5d00c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5d4b5d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d00cb4e0 con 0x7fd5d0155070 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5d4b5d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d015e840 con 0x7fd5d015b8e0 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5d4b5d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d015f2a0 con 0x7fd5d00cc820 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ce59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d00cc820 0x7fd5d01f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ce59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d00cc820 0x7fd5d01f19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:47118/0 (socket says 172.21.15.62:47118) 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ce59c640 1 -- 172.21.15.62:0/4064014857 learned_addr learned my addr 172.21.15.62:0/4064014857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ced9d640 1 --2- 172.21.15.62:0/4064014857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5d015b8e0 0x7fd5d00c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5cdd9b640 1 --2- 172.21.15.62:0/4064014857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0155070 0x7fd5d00c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ce59c640 1 -- 172.21.15.62:0/4064014857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5d015b8e0 msgr2=0x7fd5d00c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ce59c640 1 --2- 172.21.15.62:0/4064014857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5d015b8e0 0x7fd5d00c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ce59c640 1 -- 172.21.15.62:0/4064014857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0155070 msgr2=0x7fd5d00c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ce59c640 1 --2- 172.21.15.62:0/4064014857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0155070 0x7fd5d00c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ce59c640 1 -- 172.21.15.62:0/4064014857 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d0157620 con 0x7fd5d00cc820 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5ce59c640 1 --2- 172.21.15.62:0/4064014857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d00cc820 0x7fd5d01f19f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000e870 tx=0x7fd5c000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5beffd640 1 -- 172.21.15.62:0/4064014857 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0017070 con 0x7fd5d00cc820 2024-04-25T11:38:09.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5beffd640 1 -- 172.21.15.62:0/4064014857 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c000c8e0 con 0x7fd5d00cc820 2024-04-25T11:38:09.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/4064014857 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d0155a10 con 0x7fd5d00cc820 2024-04-25T11:38:09.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5beffd640 1 -- 172.21.15.62:0/4064014857 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000ccd0 con 0x7fd5d00cc820 2024-04-25T11:38:09.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.694+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/4064014857 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d0155fd0 con 0x7fd5d00cc820 2024-04-25T11:38:09.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.695+0000 7fd5beffd640 1 -- 172.21.15.62:0/4064014857 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd5c001a070 con 0x7fd5d00cc820 2024-04-25T11:38:09.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.695+0000 7fd5beffd640 1 -- 172.21.15.62:0/4064014857 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fd5c001c870 con 0x7fd5d00cc820 2024-04-25T11:38:09.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.695+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/4064014857 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d01542e0 con 0x7fd5d00cc820 2024-04-25T11:38:09.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.697+0000 7fd5beffd640 1 -- 172.21.15.62:0/4064014857 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd5d01542e0 con 0x7fd5d00cc820 2024-04-25T11:38:09.689 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:09.702+0000 7feccc419040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:38:09.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.774+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/4064014857 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "789f733e-f414-4093-9ad1-444b395333a5", "id": 2} v 0) -- 0x7fd5d00c9a70 con 0x7fd5d00cc820 2024-04-25T11:38:09.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.779+0000 7fd5beffd640 1 -- 172.21.15.62:0/4064014857 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "789f733e-f414-4093-9ad1-444b395333a5", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fd5d00c9a70 con 0x7fd5d00cc820 2024-04-25T11:38:09.766 INFO:teuthology.orchestra.run.smithi062.stdout:2 2024-04-25T11:38:09.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.780+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/4064014857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d00cc820 msgr2=0x7fd5d01f19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.780+0000 7fd5d4b5d640 1 --2- 172.21.15.62:0/4064014857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d00cc820 0x7fd5d01f19f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000e870 tx=0x7fd5c000ed30 comp rx=0 tx=0).stop 2024-04-25T11:38:09.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.781+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/4064014857 shutdown_connections 2024-04-25T11:38:09.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.781+0000 7fd5d4b5d640 1 --2- 172.21.15.62:0/4064014857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5d015b8e0 0x7fd5d00c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.781+0000 7fd5d4b5d640 1 --2- 172.21.15.62:0/4064014857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0155070 0x7fd5d00c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.781+0000 7fd5d4b5d640 1 --2- 172.21.15.62:0/4064014857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5d00cc820 0x7fd5d01f19f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.781+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/4064014857 >> 172.21.15.62:0/4064014857 conn(0x7fd5d006c680 msgr2=0x7fd5d0153080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.781+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/4064014857 shutdown_connections 2024-04-25T11:38:09.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.781+0000 7fd5d4b5d640 1 -- 172.21.15.62:0/4064014857 wait complete. 2024-04-25T11:38:09.782 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 41202350-9bfc-49ad-90f4-59c63e314ea4 3 2024-04-25T11:38:09.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.895+0000 7f481029a640 1 Processor -- start 2024-04-25T11:38:09.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.895+0000 7f481029a640 1 -- start start 2024-04-25T11:38:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f481029a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f480806bbe0 0x7f480806bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f481029a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f480806c500 0x7f480806fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f481029a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4808070280 0x7f4808073010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f481029a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f48080c5420 con 0x7f480806c500 2024-04-25T11:38:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f481029a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f48080728a0 con 0x7f480806bbe0 2024-04-25T11:38:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f481029a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4808070860 con 0x7f4808070280 2024-04-25T11:38:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e036640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f480806bbe0 0x7f480806bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e837640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4808070280 0x7f4808073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e036640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f480806bbe0 0x7f480806bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41508/0 (socket says 172.21.15.62:41508) 2024-04-25T11:38:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e036640 1 -- 172.21.15.62:0/3951333311 learned_addr learned my addr 172.21.15.62:0/3951333311 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480d835640 1 --2- 172.21.15.62:0/3951333311 >> v2:172.21.15.40:3300/0 conn(0x7f480806c500 0x7f480806fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e036640 1 -- 172.21.15.62:0/3951333311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4808070280 msgr2=0x7f4808073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e036640 1 --2- 172.21.15.62:0/3951333311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4808070280 0x7f4808073010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e036640 1 -- 172.21.15.62:0/3951333311 >> v2:172.21.15.40:3300/0 conn(0x7f480806c500 msgr2=0x7f480806fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e036640 1 --2- 172.21.15.62:0/3951333311 >> v2:172.21.15.40:3300/0 conn(0x7f480806c500 0x7f480806fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e036640 1 -- 172.21.15.62:0/3951333311 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4808070b40 con 0x7f480806bbe0 2024-04-25T11:38:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.896+0000 7f480e837640 1 --2- 172.21.15.62:0/3951333311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4808070280 0x7f4808073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.897+0000 7f480e036640 1 --2- 172.21.15.62:0/3951333311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 0x7f480806bfc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f480806c9d0 tx=0x7f480402a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe7d6577bb476d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.897+0000 7f480d034640 1 -- 172.21.15.62:0/3951333311 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804037070 con 0x7f480806bbe0 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.897+0000 7f480d034640 1 -- 172.21.15.62:0/3951333311 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4804032440 con 0x7f480806bbe0 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.897+0000 7f480d034640 1 -- 172.21.15.62:0/3951333311 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804032810 con 0x7f480806bbe0 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.898+0000 7f481029a640 1 -- 172.21.15.62:0/3951333311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 msgr2=0x7f480806bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.898+0000 7f481029a640 1 --2- 172.21.15.62:0/3951333311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 0x7f480806bfc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f480806c9d0 tx=0x7f480402a140 comp rx=0 tx=0).stop 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.898+0000 7f481029a640 1 -- 172.21.15.62:0/3951333311 shutdown_connections 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.898+0000 7f481029a640 1 --2- 172.21.15.62:0/3951333311 >> v2:172.21.15.40:3300/0 conn(0x7f480806c500 0x7f480806fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.898+0000 7f481029a640 1 --2- 172.21.15.62:0/3951333311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4808070280 0x7f4808073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.898+0000 7f481029a640 1 --2- 172.21.15.62:0/3951333311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 0x7f480806bfc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.898+0000 7f481029a640 1 -- 172.21.15.62:0/3951333311 >> 172.21.15.62:0/3951333311 conn(0x7f48080cd480 msgr2=0x7f48080cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.898+0000 7f481029a640 1 -- 172.21.15.62:0/3951333311 shutdown_connections 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.898+0000 7f481029a640 1 -- 172.21.15.62:0/3951333311 wait complete. 2024-04-25T11:38:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.899+0000 7f481029a640 1 Processor -- start 2024-04-25T11:38:09.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.899+0000 7f481029a640 1 -- start start 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f481029a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 0x7f480815b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f481029a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f480806c500 0x7f4808156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f481029a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4808070280 0x7f480815e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f481029a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4808073550 con 0x7f4808070280 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f481029a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4808071100 con 0x7f480806bbe0 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f481029a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f48080736d0 con 0x7f480806c500 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480d835640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f480806c500 0x7f4808156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480e036640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 0x7f480815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480d835640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f480806c500 0x7f4808156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:47144/0 (socket says 172.21.15.62:47144) 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480d835640 1 -- 172.21.15.62:0/625575879 learned_addr learned my addr 172.21.15.62:0/625575879 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480e837640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4808070280 0x7f480815e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480e036640 1 -- 172.21.15.62:0/625575879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f480806c500 msgr2=0x7f4808156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480e036640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f480806c500 0x7f4808156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480e036640 1 -- 172.21.15.62:0/625575879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4808070280 msgr2=0x7f480815e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480e036640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4808070280 0x7f480815e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480e036640 1 -- 172.21.15.62:0/625575879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480815bb70 con 0x7f480806bbe0 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480d835640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f480806c500 0x7f4808156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.900+0000 7f480e837640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4808070280 0x7f480815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.901+0000 7f480e036640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 0x7f480815b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4804031000 tx=0x7f48040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.901+0000 7f47feffd640 1 -- 172.21.15.62:0/625575879 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804041020 con 0x7f480806bbe0 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.901+0000 7f47feffd640 1 -- 172.21.15.62:0/625575879 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480403d660 con 0x7f480806bbe0 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.901+0000 7f481029a640 1 -- 172.21.15.62:0/625575879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48081f04a0 con 0x7f480806bbe0 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.901+0000 7f47feffd640 1 -- 172.21.15.62:0/625575879 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804037040 con 0x7f480806bbe0 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.901+0000 7f481029a640 1 -- 172.21.15.62:0/625575879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48081f0a60 con 0x7f480806bbe0 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.901+0000 7f47feffd640 1 -- 172.21.15.62:0/625575879 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4804044020 con 0x7f480806bbe0 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.901+0000 7f47feffd640 1 -- 172.21.15.62:0/625575879 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f480403b3f0 con 0x7f480806bbe0 2024-04-25T11:38:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.902+0000 7f481029a640 1 -- 172.21.15.62:0/625575879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4808069200 con 0x7f480806bbe0 2024-04-25T11:38:09.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:09.904+0000 7f47feffd640 1 -- 172.21.15.62:0/625575879 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f480404b030 con 0x7f480806bbe0 2024-04-25T11:38:09.945 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:09.958+0000 7feccc419040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:38:09.993 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:10.006+0000 7feccc419040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:38:09.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.011+0000 7f481029a640 1 -- 172.21.15.62:0/625575879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "41202350-9bfc-49ad-90f4-59c63e314ea4", "id": 3} v 0) -- 0x7f48080754d0 con 0x7f480806bbe0 2024-04-25T11:38:10.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.016+0000 7f47feffd640 1 -- 172.21.15.62:0/625575879 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "41202350-9bfc-49ad-90f4-59c63e314ea4", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f4804046400 con 0x7f480806bbe0 2024-04-25T11:38:10.004 INFO:teuthology.orchestra.run.smithi062.stdout:3 2024-04-25T11:38:10.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.017+0000 7f481029a640 1 -- 172.21.15.62:0/625575879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 msgr2=0x7f480815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.017+0000 7f481029a640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 0x7f480815b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4804031000 tx=0x7f48040046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.017+0000 7f481029a640 1 -- 172.21.15.62:0/625575879 shutdown_connections 2024-04-25T11:38:10.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.017+0000 7f481029a640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4808070280 0x7f480815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.017+0000 7f481029a640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f480806c500 0x7f4808156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.017+0000 7f481029a640 1 --2- 172.21.15.62:0/625575879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f480806bbe0 0x7f480815b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.017+0000 7f481029a640 1 -- 172.21.15.62:0/625575879 >> 172.21.15.62:0/625575879 conn(0x7f48080cd480 msgr2=0x7f48080cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.017+0000 7f481029a640 1 -- 172.21.15.62:0/625575879 shutdown_connections 2024-04-25T11:38:10.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.018+0000 7f481029a640 1 -- 172.21.15.62:0/625575879 wait complete. 2024-04-25T11:38:10.008 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:10.021+0000 7f5bbbf21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:38:10.015 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new ba3ad959-4d4a-48ab-8e7d-a2a9151280ba 4 2024-04-25T11:38:10.104 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:10.116+0000 7f5bbbf21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:38:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.129+0000 7fd948279640 1 Processor -- start 2024-04-25T11:38:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.129+0000 7fd948279640 1 -- start start 2024-04-25T11:38:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd948279640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd940071a20 0x7fd9400c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd948279640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9400cab70 0x7fd940154d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd948279640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94006d4a0 con 0x7fd940071e00 2024-04-25T11:38:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd948279640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd9400c9480 con 0x7fd9400cab70 2024-04-25T11:38:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd948279640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd94006e0b0 con 0x7fd940071a20 2024-04-25T11:38:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd946015640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd940071a20 0x7fd9400c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd945814640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd940071e00 0x7fd94015e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:46418/0 (socket says 172.21.15.62:46418) 2024-04-25T11:38:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd946015640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd940071a20 0x7fd9400c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41524/0 (socket says 172.21.15.62:41524) 2024-04-25T11:38:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd945814640 1 -- 172.21.15.62:0/1131783493 learned_addr learned my addr 172.21.15.62:0/1131783493 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T11:38:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd946816640 1 --2- 172.21.15.62:0/1131783493 >> v2:172.21.15.40:3300/0 conn(0x7fd9400cab70 0x7fd940154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd946015640 1 -- 172.21.15.62:0/1131783493 >> v2:172.21.15.40:3300/0 conn(0x7fd9400cab70 msgr2=0x7fd940154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd946015640 1 --2- 172.21.15.62:0/1131783493 >> v2:172.21.15.40:3300/0 conn(0x7fd9400cab70 0x7fd940154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd946015640 1 -- 172.21.15.62:0/1131783493 >> v1:172.21.15.62:6789/0 conn(0x7fd940071e00 legacy=0x7fd94015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd946015640 1 -- 172.21.15.62:0/1131783493 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94006d780 con 0x7fd940071a20 2024-04-25T11:38:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.130+0000 7fd945013640 1 -- 172.21.15.62:0/1131783493 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 910301581 0 0) 0x7fd94006d4a0 con 0x7fd940071e00 2024-04-25T11:38:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.131+0000 7fd946015640 1 --2- 172.21.15.62:0/1131783493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd940071a20 0x7fd9400c8f40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd9400cf600 tx=0x7fd93c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=967b7f92b1fd7f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.131+0000 7fd945013640 1 -- 172.21.15.62:0/1131783493 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c037070 con 0x7fd940071a20 2024-04-25T11:38:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.131+0000 7fd945013640 1 -- 172.21.15.62:0/1131783493 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c032440 con 0x7fd940071a20 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.131+0000 7fd945013640 1 -- 172.21.15.62:0/1131783493 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c032810 con 0x7fd940071a20 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.132+0000 7fd948279640 1 -- 172.21.15.62:0/1131783493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd940071a20 msgr2=0x7fd9400c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.132+0000 7fd948279640 1 --2- 172.21.15.62:0/1131783493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd940071a20 0x7fd9400c8f40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd9400cf600 tx=0x7fd93c02a230 comp rx=0 tx=0).stop 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.132+0000 7fd948279640 1 -- 172.21.15.62:0/1131783493 shutdown_connections 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.132+0000 7fd948279640 1 --2- 172.21.15.62:0/1131783493 >> v2:172.21.15.40:3300/0 conn(0x7fd9400cab70 0x7fd940154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.132+0000 7fd948279640 1 --2- 172.21.15.62:0/1131783493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd940071a20 0x7fd9400c8f40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.132+0000 7fd948279640 1 -- 172.21.15.62:0/1131783493 >> 172.21.15.62:0/1131783493 conn(0x7fd94006b970 msgr2=0x7fd94006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.132+0000 7fd948279640 1 -- 172.21.15.62:0/1131783493 shutdown_connections 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.132+0000 7fd948279640 1 -- 172.21.15.62:0/1131783493 wait complete. 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.133+0000 7fd948279640 1 Processor -- start 2024-04-25T11:38:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.133+0000 7fd948279640 1 -- start start 2024-04-25T11:38:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd948279640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd940071a20 0x7fd9400c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd948279640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd940071e00 0x7fd94015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd948279640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9400cab70 0x7fd9400cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd948279640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd94015ef20 con 0x7fd940071a20 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd948279640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd94006dd40 con 0x7fd9400cab70 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd948279640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd94015f0a0 con 0x7fd940071e00 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946816640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9400cab70 0x7fd9400cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd945814640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd940071e00 0x7fd94015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946816640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9400cab70 0x7fd9400cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41538/0 (socket says 172.21.15.62:41538) 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946816640 1 -- 172.21.15.62:0/2373524616 learned_addr learned my addr 172.21.15.62:0/2373524616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946015640 1 --2- 172.21.15.62:0/2373524616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd940071a20 0x7fd9400c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946816640 1 -- 172.21.15.62:0/2373524616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd940071e00 msgr2=0x7fd94015a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946816640 1 --2- 172.21.15.62:0/2373524616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd940071e00 0x7fd94015a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946816640 1 -- 172.21.15.62:0/2373524616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd940071a20 msgr2=0x7fd9400c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946816640 1 --2- 172.21.15.62:0/2373524616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd940071a20 0x7fd9400c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946816640 1 -- 172.21.15.62:0/2373524616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9400c8a80 con 0x7fd9400cab70 2024-04-25T11:38:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.134+0000 7fd946816640 1 --2- 172.21.15.62:0/2373524616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9400cab70 0x7fd9400cebb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd938009d30 tx=0x7fd93800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.135+0000 7fd936ffd640 1 -- 172.21.15.62:0/2373524616 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938018020 con 0x7fd9400cab70 2024-04-25T11:38:10.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.135+0000 7fd936ffd640 1 -- 172.21.15.62:0/2373524616 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd938014480 con 0x7fd9400cab70 2024-04-25T11:38:10.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.135+0000 7fd936ffd640 1 -- 172.21.15.62:0/2373524616 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938014870 con 0x7fd9400cab70 2024-04-25T11:38:10.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.135+0000 7fd948279640 1 -- 172.21.15.62:0/2373524616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9400ccf60 con 0x7fd9400cab70 2024-04-25T11:38:10.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.135+0000 7fd948279640 1 -- 172.21.15.62:0/2373524616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9400cd520 con 0x7fd9400cab70 2024-04-25T11:38:10.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.135+0000 7fd936ffd640 1 -- 172.21.15.62:0/2373524616 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd938005d10 con 0x7fd9400cab70 2024-04-25T11:38:10.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.135+0000 7fd936ffd640 1 -- 172.21.15.62:0/2373524616 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fd9380138f0 con 0x7fd9400cab70 2024-04-25T11:38:10.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.136+0000 7fd948279640 1 -- 172.21.15.62:0/2373524616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd940069200 con 0x7fd9400cab70 2024-04-25T11:38:10.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.138+0000 7fd936ffd640 1 -- 172.21.15.62:0/2373524616 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd938022070 con 0x7fd9400cab70 2024-04-25T11:38:10.150 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:10.163+0000 7f5bbbf21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:38:10.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.211+0000 7fd948279640 1 -- 172.21.15.62:0/2373524616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ba3ad959-4d4a-48ab-8e7d-a2a9151280ba", "id": 4} v 0) -- 0x7fd9400c9830 con 0x7fd9400cab70 2024-04-25T11:38:10.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.216+0000 7fd936ffd640 1 -- 172.21.15.62:0/2373524616 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba3ad959-4d4a-48ab-8e7d-a2a9151280ba", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fd938010040 con 0x7fd9400cab70 2024-04-25T11:38:10.204 INFO:teuthology.orchestra.run.smithi062.stdout:4 2024-04-25T11:38:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.217+0000 7fd948279640 1 -- 172.21.15.62:0/2373524616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9400cab70 msgr2=0x7fd9400cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.217+0000 7fd948279640 1 --2- 172.21.15.62:0/2373524616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9400cab70 0x7fd9400cebb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd938009d30 tx=0x7fd93800c490 comp rx=0 tx=0).stop 2024-04-25T11:38:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.217+0000 7fd948279640 1 -- 172.21.15.62:0/2373524616 shutdown_connections 2024-04-25T11:38:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.217+0000 7fd948279640 1 --2- 172.21.15.62:0/2373524616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9400cab70 0x7fd9400cebb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.217+0000 7fd948279640 1 --2- 172.21.15.62:0/2373524616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd940071e00 0x7fd94015a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.217+0000 7fd948279640 1 --2- 172.21.15.62:0/2373524616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd940071a20 0x7fd9400c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.217+0000 7fd948279640 1 -- 172.21.15.62:0/2373524616 >> 172.21.15.62:0/2373524616 conn(0x7fd94006b970 msgr2=0x7fd9400c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.217+0000 7fd948279640 1 -- 172.21.15.62:0/2373524616 shutdown_connections 2024-04-25T11:38:10.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.217+0000 7fd948279640 1 -- 172.21.15.62:0/2373524616 wait complete. 2024-04-25T11:38:10.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new bd3dbba0-ba11-4cd2-b930-d884380986b1 5 2024-04-25T11:38:10.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.329+0000 7efdff1f7640 1 Processor -- start 2024-04-25T11:38:10.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.329+0000 7efdff1f7640 1 -- start start 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdff1f7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdf8072c00 0x7efdf8073020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdff1f7640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf80c8af0 0x7efdf80c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdff1f7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf80c94c0 con 0x7efdf8155090 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdff1f7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efdf80c5590 con 0x7efdf8072c00 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdff1f7640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efdf815e7f0 con 0x7efdf80c8af0 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdfd794640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf80c8af0 0x7efdf80c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdfd794640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf80c8af0 0x7efdf80c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:47168/0 (socket says 172.21.15.62:47168) 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdfd794640 1 -- 172.21.15.62:0/1713823279 learned_addr learned my addr 172.21.15.62:0/1713823279 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdf7fff640 1 --2- 172.21.15.62:0/1713823279 >> v2:172.21.15.40:3300/0 conn(0x7efdf8072c00 0x7efdf8073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdf77fe640 1 -- 172.21.15.62:0/1713823279 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2471092050 0 0) 0x7efdf80c94c0 con 0x7efdf8155090 2024-04-25T11:38:10.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdfd794640 1 -- 172.21.15.62:0/1713823279 >> v2:172.21.15.40:3300/0 conn(0x7efdf8072c00 msgr2=0x7efdf8073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.330+0000 7efdfd794640 1 --2- 172.21.15.62:0/1713823279 >> v2:172.21.15.40:3300/0 conn(0x7efdf8072c00 0x7efdf8073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.331+0000 7efdfd794640 1 -- 172.21.15.62:0/1713823279 >> v1:172.21.15.40:6789/0 conn(0x7efdf8155090 legacy=0x7efdf80cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.331+0000 7efdfd794640 1 -- 172.21.15.62:0/1713823279 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf80c8610 con 0x7efdf80c8af0 2024-04-25T11:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.331+0000 7efdfd794640 1 --2- 172.21.15.62:0/1713823279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf80c8af0 0x7efdf80c8f80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009e50 tx=0x7efdf002a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0b034a31de2172 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.331+0000 7efdf77fe640 1 -- 172.21.15.62:0/1713823279 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0036070 con 0x7efdf80c8af0 2024-04-25T11:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.331+0000 7efdf77fe640 1 -- 172.21.15.62:0/1713823279 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf0031600 con 0x7efdf80c8af0 2024-04-25T11:38:10.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.331+0000 7efdf77fe640 1 -- 172.21.15.62:0/1713823279 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00319d0 con 0x7efdf80c8af0 2024-04-25T11:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.332+0000 7efdff1f7640 1 -- 172.21.15.62:0/1713823279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf80c8af0 msgr2=0x7efdf80c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.332+0000 7efdff1f7640 1 --2- 172.21.15.62:0/1713823279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf80c8af0 0x7efdf80c8f80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009e50 tx=0x7efdf002a7a0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.332+0000 7efdff1f7640 1 -- 172.21.15.62:0/1713823279 shutdown_connections 2024-04-25T11:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.332+0000 7efdff1f7640 1 --2- 172.21.15.62:0/1713823279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf80c8af0 0x7efdf80c8f80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.332+0000 7efdff1f7640 1 --2- 172.21.15.62:0/1713823279 >> v2:172.21.15.40:3300/0 conn(0x7efdf8072c00 0x7efdf8073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.332+0000 7efdff1f7640 1 -- 172.21.15.62:0/1713823279 >> 172.21.15.62:0/1713823279 conn(0x7efdf80703b0 msgr2=0x7efdf8070790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.332+0000 7efdff1f7640 1 -- 172.21.15.62:0/1713823279 shutdown_connections 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.332+0000 7efdff1f7640 1 -- 172.21.15.62:0/1713823279 wait complete. 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.333+0000 7efdff1f7640 1 Processor -- start 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.333+0000 7efdff1f7640 1 -- start start 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdff1f7640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf8072c00 0x7efdf815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdff1f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdf80c8af0 0x7efdf814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdff1f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdf8155090 0x7efdf81481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdff1f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdf815f030 con 0x7efdf8155090 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdff1f7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efdf815ed50 con 0x7efdf80c8af0 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdff1f7640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efdf815f310 con 0x7efdf8072c00 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdf7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdf80c8af0 0x7efdf814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdfcf93640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf8072c00 0x7efdf815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdf7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdf80c8af0 0x7efdf814beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41548/0 (socket says 172.21.15.62:41548) 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdf7fff640 1 -- 172.21.15.62:0/1293826216 learned_addr learned my addr 172.21.15.62:0/1293826216 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.334+0000 7efdfd794640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdf8155090 0x7efdf81481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdfd794640 1 -- 172.21.15.62:0/1293826216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf8072c00 msgr2=0x7efdf815a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdfd794640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf8072c00 0x7efdf815a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdfd794640 1 -- 172.21.15.62:0/1293826216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdf80c8af0 msgr2=0x7efdf814beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdfd794640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdf80c8af0 0x7efdf814beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdfd794640 1 -- 172.21.15.62:0/1293826216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf814c6f0 con 0x7efdf8155090 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdfcf93640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf8072c00 0x7efdf815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdf7fff640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdf80c8af0 0x7efdf814beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdfd794640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdf8155090 0x7efdf81481c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efdf0030000 tx=0x7efdf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdf57fa640 1 -- 172.21.15.62:0/1293826216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0036050 con 0x7efdf8155090 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdf57fa640 1 -- 172.21.15.62:0/1293826216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf0042070 con 0x7efdf8155090 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdf57fa640 1 -- 172.21.15.62:0/1293826216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0009510 con 0x7efdf8155090 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdff1f7640 1 -- 172.21.15.62:0/1293826216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf815a920 con 0x7efdf8155090 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdff1f7640 1 -- 172.21.15.62:0/1293826216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf815aea0 con 0x7efdf8155090 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdf57fa640 1 -- 172.21.15.62:0/1293826216 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efdf0007b70 con 0x7efdf8155090 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.335+0000 7efdf57fa640 1 -- 172.21.15.62:0/1293826216 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7efdf0047af0 con 0x7efdf8155090 2024-04-25T11:38:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.336+0000 7efdff1f7640 1 -- 172.21.15.62:0/1293826216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf8069200 con 0x7efdf8155090 2024-04-25T11:38:10.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.338+0000 7efdf57fa640 1 -- 172.21.15.62:0/1293826216 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efdf0047460 con 0x7efdf8155090 2024-04-25T11:38:10.338 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:10.351+0000 7f5bbbf21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:38:10.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.412+0000 7efdff1f7640 1 -- 172.21.15.62:0/1293826216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bd3dbba0-ba11-4cd2-b930-d884380986b1", "id": 5} v 0) -- 0x7efdf814ab80 con 0x7efdf8155090 2024-04-25T11:38:10.400 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:10.412+0000 7f5bbbf21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:38:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.415+0000 7efdf57fa640 1 -- 172.21.15.62:0/1293826216 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bd3dbba0-ba11-4cd2-b930-d884380986b1", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7efdf0047640 con 0x7efdf8155090 2024-04-25T11:38:10.403 INFO:teuthology.orchestra.run.smithi062.stdout:5 2024-04-25T11:38:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.416+0000 7efdff1f7640 1 -- 172.21.15.62:0/1293826216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdf8155090 msgr2=0x7efdf81481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.416+0000 7efdff1f7640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdf8155090 0x7efdf81481c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efdf0030000 tx=0x7efdf0004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.417+0000 7efdff1f7640 1 -- 172.21.15.62:0/1293826216 shutdown_connections 2024-04-25T11:38:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.417+0000 7efdff1f7640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdf8155090 0x7efdf81481c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.417+0000 7efdff1f7640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdf80c8af0 0x7efdf814beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.417+0000 7efdff1f7640 1 --2- 172.21.15.62:0/1293826216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdf8072c00 0x7efdf815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.417+0000 7efdff1f7640 1 -- 172.21.15.62:0/1293826216 >> 172.21.15.62:0/1293826216 conn(0x7efdf80703b0 msgr2=0x7efdf806ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.417+0000 7efdff1f7640 1 -- 172.21.15.62:0/1293826216 shutdown_connections 2024-04-25T11:38:10.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.417+0000 7efdff1f7640 1 -- 172.21.15.62:0/1293826216 wait complete. 2024-04-25T11:38:10.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new c97e06d6-a79a-4ff1-a5ae-404ae70f8638 6 2024-04-25T11:38:10.458 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:10.471+0000 7f5bbbf21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:38:10.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.527+0000 7f3ddcafa640 1 Processor -- start 2024-04-25T11:38:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.527+0000 7f3ddcafa640 1 -- start start 2024-04-25T11:38:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.528+0000 7f3ddcafa640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8151e70 0x7f3dd8158540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.528+0000 7f3ddcafa640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd8157d80 con 0x7f3dd80c7c80 2024-04-25T11:38:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.528+0000 7f3ddcafa640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd8152250 con 0x7f3dd8070420 2024-04-25T11:38:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.528+0000 7f3ddcafa640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd80c7040 con 0x7f3dd8151e70 2024-04-25T11:38:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.529+0000 7f3dd659c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8151e70 0x7f3dd8158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.529+0000 7f3dd6d9d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3dd8070420 0x7f3dd81624f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:46430/0 (socket says 172.21.15.62:46430) 2024-04-25T11:38:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.529+0000 7f3dd6d9d640 1 -- 172.21.15.62:0/1175536432 learned_addr learned my addr 172.21.15.62:0/1175536432 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T11:38:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.529+0000 7f3dd559a640 1 -- 172.21.15.62:0/1175536432 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992141639 0 0) 0x7f3dd8152250 con 0x7f3dd8070420 2024-04-25T11:38:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.529+0000 7f3dd659c640 1 -- 172.21.15.62:0/1175536432 >> v1:172.21.15.62:6789/0 conn(0x7f3dd8070420 legacy=0x7f3dd81624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.529+0000 7f3dd659c640 1 -- 172.21.15.62:0/1175536432 >> v1:172.21.15.40:6789/0 conn(0x7f3dd80c7c80 legacy=0x7f3dd80c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.529+0000 7f3dd659c640 1 -- 172.21.15.62:0/1175536432 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd80c67a0 con 0x7f3dd8151e70 2024-04-25T11:38:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.530+0000 7f3dd659c640 1 --2- 172.21.15.62:0/1175536432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8151e70 0x7f3dd8158540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0097d0 tx=0x7f3dcc02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5b4df4b748ee57d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.530+0000 7f3dd559a640 1 -- 172.21.15.62:0/1175536432 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc037070 con 0x7f3dd8151e70 2024-04-25T11:38:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.530+0000 7f3dd559a640 1 -- 172.21.15.62:0/1175536432 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc032440 con 0x7f3dd8151e70 2024-04-25T11:38:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.530+0000 7f3dd559a640 1 -- 172.21.15.62:0/1175536432 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc0327f0 con 0x7f3dd8151e70 2024-04-25T11:38:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.531+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1175536432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8151e70 msgr2=0x7f3dd8158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.531+0000 7f3ddcafa640 1 --2- 172.21.15.62:0/1175536432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8151e70 0x7f3dd8158540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0097d0 tx=0x7f3dcc02a290 comp rx=0 tx=0).stop 2024-04-25T11:38:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.531+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1175536432 shutdown_connections 2024-04-25T11:38:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.531+0000 7f3ddcafa640 1 --2- 172.21.15.62:0/1175536432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8151e70 0x7f3dd8158540 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.531+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1175536432 >> 172.21.15.62:0/1175536432 conn(0x7f3dd80676b0 msgr2=0x7f3dd8067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.531+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1175536432 shutdown_connections 2024-04-25T11:38:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.531+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1175536432 wait complete. 2024-04-25T11:38:10.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.533+0000 7f3ddcafa640 1 Processor -- start 2024-04-25T11:38:10.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.533+0000 7f3ddcafa640 1 -- start start 2024-04-25T11:38:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3ddcafa640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8070420 0x7f3dd80cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3ddcafa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd80c7c80 0x7f3dd81f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3ddcafa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd8151e70 0x7f3dd81ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3ddcafa640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd8158020 con 0x7f3dd80c7c80 2024-04-25T11:38:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3ddcafa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd80c6d60 con 0x7f3dd8151e70 2024-04-25T11:38:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3ddcafa640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd8158a80 con 0x7f3dd8070420 2024-04-25T11:38:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3dd659c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8070420 0x7f3dd80cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3dd659c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8070420 0x7f3dd80cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:47198/0 (socket says 172.21.15.62:47198) 2024-04-25T11:38:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3dd659c640 1 -- 172.21.15.62:0/1498416272 learned_addr learned my addr 172.21.15.62:0/1498416272 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3dd6d9d640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd8151e70 0x7f3dd81ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3dd5d9b640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd80c7c80 0x7f3dd81f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3dd659c640 1 -- 172.21.15.62:0/1498416272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd8151e70 msgr2=0x7f3dd81ef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.534+0000 7f3dd659c640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd8151e70 0x7f3dd81ef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dd659c640 1 -- 172.21.15.62:0/1498416272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd80c7c80 msgr2=0x7f3dd81f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dd659c640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd80c7c80 0x7f3dd81f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dd659c640 1 -- 172.21.15.62:0/1498416272 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd81ef1f0 con 0x7f3dd8070420 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dd6d9d640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd8151e70 0x7f3dd81ef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dd659c640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8070420 0x7f3dd80cc650 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc035040 tx=0x7f3dcc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dc6ffd640 1 -- 172.21.15.62:0/1498416272 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc037050 con 0x7f3dd8070420 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dc6ffd640 1 -- 172.21.15.62:0/1498416272 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc038040 con 0x7f3dd8070420 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1498416272 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd81f2900 con 0x7f3dd8070420 2024-04-25T11:38:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dc6ffd640 1 -- 172.21.15.62:0/1498416272 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc004490 con 0x7f3dd8070420 2024-04-25T11:38:10.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1498416272 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd806c060 con 0x7f3dd8070420 2024-04-25T11:38:10.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dc6ffd640 1 -- 172.21.15.62:0/1498416272 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3dcc030900 con 0x7f3dd8070420 2024-04-25T11:38:10.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.535+0000 7f3dc6ffd640 1 -- 172.21.15.62:0/1498416272 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f3dcc003ac0 con 0x7f3dd8070420 2024-04-25T11:38:10.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.536+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1498416272 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd8157240 con 0x7f3dd8070420 2024-04-25T11:38:10.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.540+0000 7f3dc6ffd640 1 -- 172.21.15.62:0/1498416272 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3dd8157240 con 0x7f3dd8070420 2024-04-25T11:38:10.585 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:10.597+0000 7f5bbbf21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.614+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1498416272 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c97e06d6-a79a-4ff1-a5ae-404ae70f8638", "id": 6} v 0) -- 0x7f3dd8069200 con 0x7f3dd8070420 2024-04-25T11:38:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.622+0000 7f3dc6ffd640 1 -- 172.21.15.62:0/1498416272 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c97e06d6-a79a-4ff1-a5ae-404ae70f8638", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f3dcc030dd0 con 0x7f3dd8070420 2024-04-25T11:38:10.610 INFO:teuthology.orchestra.run.smithi062.stdout:6 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.623+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1498416272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8070420 msgr2=0x7f3dd80cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.623+0000 7f3ddcafa640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8070420 0x7f3dd80cc650 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc035040 tx=0x7f3dcc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.623+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1498416272 shutdown_connections 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.623+0000 7f3ddcafa640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3dd8151e70 0x7f3dd81ef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.623+0000 7f3ddcafa640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd80c7c80 0x7f3dd81f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.623+0000 7f3ddcafa640 1 --2- 172.21.15.62:0/1498416272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd8070420 0x7f3dd80cc650 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.623+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1498416272 >> 172.21.15.62:0/1498416272 conn(0x7f3dd80676b0 msgr2=0x7f3dd80c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.623+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1498416272 shutdown_connections 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.623+0000 7f3ddcafa640 1 -- 172.21.15.62:0/1498416272 wait complete. 2024-04-25T11:38:10.621 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 83214518-6405-4702-8877-900f7b544f9f 7 2024-04-25T11:38:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.733+0000 7fe891995640 1 Processor -- start 2024-04-25T11:38:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.733+0000 7fe891995640 1 -- start start 2024-04-25T11:38:10.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe891995640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c1514b0 0x7fe88c0c2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe891995640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe88c0c1eb0 0x7fe88c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe891995640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe88c0c8cf0 con 0x7fe88c0c22d0 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe891995640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fe88c157bd0 con 0x7fe88c0c1eb0 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe891995640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c073fc0 con 0x7fe88c1514b0 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe88affd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c1514b0 0x7fe88c0c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe883fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe88c0c1eb0 0x7fe88c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe88affd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c1514b0 0x7fe88c0c2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:47210/0 (socket says 172.21.15.62:47210) 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe88affd640 1 -- 172.21.15.62:0/731037141 learned_addr learned my addr 172.21.15.62:0/731037141 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe88a7fc640 1 -- 172.21.15.62:0/731037141 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 885223174 0 0) 0x7fe88c0c8cf0 con 0x7fe88c0c22d0 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe88a7fc640 1 -- 172.21.15.62:0/731037141 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe878003520 con 0x7fe88c0c22d0 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe883fff640 1 -- 172.21.15.62:0/731037141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c1514b0 msgr2=0x7fe88c0c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe883fff640 1 --2- 172.21.15.62:0/731037141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c1514b0 0x7fe88c0c2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe883fff640 1 -- 172.21.15.62:0/731037141 >> v1:172.21.15.62:6789/0 conn(0x7fe88c0c22d0 legacy=0x7fe88c15a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe883fff640 1 -- 172.21.15.62:0/731037141 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c0c8fd0 con 0x7fe88c0c1eb0 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.735+0000 7fe88affd640 1 --2- 172.21.15.62:0/731037141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c1514b0 0x7fe88c0c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.736+0000 7fe883fff640 1 --2- 172.21.15.62:0/731037141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c0c1eb0 0x7fe88c0c7c30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe87400b850 tx=0x7fe87402a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960d733341d90cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.736+0000 7fe88a7fc640 1 -- 172.21.15.62:0/731037141 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87402f710 con 0x7fe88c0c1eb0 2024-04-25T11:38:10.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.736+0000 7fe88a7fc640 1 -- 172.21.15.62:0/731037141 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87402f8b0 con 0x7fe88c0c1eb0 2024-04-25T11:38:10.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.736+0000 7fe88a7fc640 1 -- 172.21.15.62:0/731037141 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87402fc80 con 0x7fe88c0c1eb0 2024-04-25T11:38:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.737+0000 7fe891995640 1 -- 172.21.15.62:0/731037141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c0c1eb0 msgr2=0x7fe88c0c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.737+0000 7fe891995640 1 --2- 172.21.15.62:0/731037141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c0c1eb0 0x7fe88c0c7c30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe87400b850 tx=0x7fe87402a620 comp rx=0 tx=0).stop 2024-04-25T11:38:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.737+0000 7fe891995640 1 -- 172.21.15.62:0/731037141 shutdown_connections 2024-04-25T11:38:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.737+0000 7fe891995640 1 --2- 172.21.15.62:0/731037141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c0c1eb0 0x7fe88c0c7c30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.737+0000 7fe891995640 1 --2- 172.21.15.62:0/731037141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c1514b0 0x7fe88c0c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.737+0000 7fe891995640 1 -- 172.21.15.62:0/731037141 >> 172.21.15.62:0/731037141 conn(0x7fe88c0c9650 msgr2=0x7fe88c0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.737+0000 7fe891995640 1 -- 172.21.15.62:0/731037141 shutdown_connections 2024-04-25T11:38:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.737+0000 7fe891995640 1 -- 172.21.15.62:0/731037141 wait complete. 2024-04-25T11:38:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.738+0000 7fe891995640 1 Processor -- start 2024-04-25T11:38:10.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.738+0000 7fe891995640 1 -- start start 2024-04-25T11:38:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe891995640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c0c1eb0 0x7fe88c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe891995640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0c22d0 0x7fe88c14cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe891995640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c1514b0 0x7fe88c14dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe891995640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c157250 con 0x7fe88c0c22d0 2024-04-25T11:38:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe891995640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c157860 con 0x7fe88c1514b0 2024-04-25T11:38:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe891995640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c0b2f40 con 0x7fe88c0c1eb0 2024-04-25T11:38:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88affd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c0c1eb0 0x7fe88c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c1514b0 0x7fe88c14dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88affd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c0c1eb0 0x7fe88c156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:47214/0 (socket says 172.21.15.62:47214) 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88affd640 1 -- 172.21.15.62:0/2849474071 learned_addr learned my addr 172.21.15.62:0/2849474071 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c1514b0 0x7fe88c14dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41568/0 (socket says 172.21.15.62:41568) 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe883fff640 1 --2- 172.21.15.62:0/2849474071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0c22d0 0x7fe88c14cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88b7fe640 1 -- 172.21.15.62:0/2849474071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c0c1eb0 msgr2=0x7fe88c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88b7fe640 1 --2- 172.21.15.62:0/2849474071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c0c1eb0 0x7fe88c156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88b7fe640 1 -- 172.21.15.62:0/2849474071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0c22d0 msgr2=0x7fe88c14cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88b7fe640 1 --2- 172.21.15.62:0/2849474071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0c22d0 0x7fe88c14cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88b7fe640 1 -- 172.21.15.62:0/2849474071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c154270 con 0x7fe88c1514b0 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.739+0000 7fe88affd640 1 --2- 172.21.15.62:0/2849474071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c0c1eb0 0x7fe88c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.740+0000 7fe88b7fe640 1 --2- 172.21.15.62:0/2849474071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c1514b0 0x7fe88c14dec0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c6f0 tx=0x7fe88400cbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.740+0000 7fe888ff9640 1 -- 172.21.15.62:0/2849474071 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884003970 con 0x7fe88c1514b0 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.740+0000 7fe888ff9640 1 -- 172.21.15.62:0/2849474071 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe884003b30 con 0x7fe88c1514b0 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.740+0000 7fe888ff9640 1 -- 172.21.15.62:0/2849474071 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88400a850 con 0x7fe88c1514b0 2024-04-25T11:38:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.740+0000 7fe891995640 1 -- 172.21.15.62:0/2849474071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c154550 con 0x7fe88c1514b0 2024-04-25T11:38:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.740+0000 7fe891995640 1 -- 172.21.15.62:0/2849474071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c14c7e0 con 0x7fe88c1514b0 2024-04-25T11:38:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.740+0000 7fe888ff9640 1 -- 172.21.15.62:0/2849474071 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe88400a9f0 con 0x7fe88c1514b0 2024-04-25T11:38:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.740+0000 7fe888ff9640 1 -- 172.21.15.62:0/2849474071 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fe88400ace0 con 0x7fe88c1514b0 2024-04-25T11:38:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.741+0000 7fe891995640 1 -- 172.21.15.62:0/2849474071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c069200 con 0x7fe88c1514b0 2024-04-25T11:38:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.743+0000 7fe888ff9640 1 -- 172.21.15.62:0/2849474071 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe8840050d0 con 0x7fe88c1514b0 2024-04-25T11:38:10.731 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:10.744+0000 7feccc419040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:38:10.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.817+0000 7fe891995640 1 -- 172.21.15.62:0/2849474071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "83214518-6405-4702-8877-900f7b544f9f", "id": 7} v 0) -- 0x7fe88c0c8770 con 0x7fe88c1514b0 2024-04-25T11:38:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.822+0000 7fe888ff9640 1 -- 172.21.15.62:0/2849474071 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "83214518-6405-4702-8877-900f7b544f9f", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fe8840052b0 con 0x7fe88c1514b0 2024-04-25T11:38:10.810 INFO:teuthology.orchestra.run.smithi062.stdout:7 2024-04-25T11:38:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.823+0000 7fe891995640 1 -- 172.21.15.62:0/2849474071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c1514b0 msgr2=0x7fe88c14dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.823+0000 7fe891995640 1 --2- 172.21.15.62:0/2849474071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c1514b0 0x7fe88c14dec0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c6f0 tx=0x7fe88400cbb0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.823+0000 7fe891995640 1 -- 172.21.15.62:0/2849474071 shutdown_connections 2024-04-25T11:38:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.823+0000 7fe891995640 1 --2- 172.21.15.62:0/2849474071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88c1514b0 0x7fe88c14dec0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.823+0000 7fe891995640 1 --2- 172.21.15.62:0/2849474071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe88c0c22d0 0x7fe88c14cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.823+0000 7fe891995640 1 --2- 172.21.15.62:0/2849474071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe88c0c1eb0 0x7fe88c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.823+0000 7fe891995640 1 -- 172.21.15.62:0/2849474071 >> 172.21.15.62:0/2849474071 conn(0x7fe88c0c9650 msgr2=0x7fe88c1486e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.824+0000 7fe891995640 1 -- 172.21.15.62:0/2849474071 shutdown_connections 2024-04-25T11:38:10.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T11:38:10.824+0000 7fe891995640 1 -- 172.21.15.62:0/2849474071 wait complete. 2024-04-25T11:38:10.821 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T11:38:10.821 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T11:38:10.824 INFO:tasks.ceph.osd.0:Started 2024-04-25T11:38:10.824 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T11:38:10.824 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T11:38:10.824 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:10.836+0000 7feccc419040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:38:10.826 INFO:tasks.ceph.osd.1:Started 2024-04-25T11:38:10.826 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T11:38:10.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T11:38:10.828 INFO:tasks.ceph.osd.2:Started 2024-04-25T11:38:10.828 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T11:38:10.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T11:38:10.831 INFO:tasks.ceph.osd.3:Started 2024-04-25T11:38:10.831 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T11:38:10.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T11:38:10.833 INFO:tasks.ceph.osd.4:Started 2024-04-25T11:38:10.833 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T11:38:10.833 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T11:38:10.835 INFO:tasks.ceph.osd.5:Started 2024-04-25T11:38:10.835 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T11:38:10.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T11:38:10.837 INFO:tasks.ceph.osd.6:Started 2024-04-25T11:38:10.838 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T11:38:10.838 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T11:38:10.840 INFO:tasks.ceph.osd.7:Started 2024-04-25T11:38:10.840 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:10.882 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:10.895+0000 7feccc419040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:38:10.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.981+0000 7fd8e7c2a640 1 Processor -- start 2024-04-25T11:38:10.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.981+0000 7fd8e7c2a640 1 -- start start 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e7c2a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8e0073110 0x7fd8e00c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e7c2a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e0073bc0 con 0x7fd8e0151e40 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e7c2a640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e0070d70 con 0x7fd8e0151a60 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e7c2a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e0071050 con 0x7fd8e0073110 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e51c5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd8e0151e40 0x7fd8e00c8370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41144/0 (socket says 172.21.15.40:41144) 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e51c5640 1 -- 172.21.15.40:0/3398590593 learned_addr learned my addr 172.21.15.40:0/3398590593 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e61c7640 1 --2- 172.21.15.40:0/3398590593 >> v2:172.21.15.62:3300/0 conn(0x7fd8e0073110 0x7fd8e00c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e49c4640 1 -- 172.21.15.40:0/3398590593 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 689369416 0 0) 0x7fd8e0070d70 con 0x7fd8e0151a60 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e49c4640 1 -- 172.21.15.40:0/3398590593 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8cc003520 con 0x7fd8e0151a60 2024-04-25T11:38:10.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e49c4640 1 -- 172.21.15.40:0/3398590593 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823796767 0 0) 0x7fd8e0073bc0 con 0x7fd8e0151e40 2024-04-25T11:38:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e49c4640 1 -- 172.21.15.40:0/3398590593 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8e0070d70 con 0x7fd8e0151e40 2024-04-25T11:38:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e61c7640 1 -- 172.21.15.40:0/3398590593 >> v1:172.21.15.62:6789/0 conn(0x7fd8e0151a60 legacy=0x7fd8e00c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e61c7640 1 -- 172.21.15.40:0/3398590593 >> v1:172.21.15.40:6789/0 conn(0x7fd8e0151e40 legacy=0x7fd8e00c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.982+0000 7fd8e61c7640 1 -- 172.21.15.40:0/3398590593 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e0073570 con 0x7fd8e0073110 2024-04-25T11:38:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.983+0000 7fd8e61c7640 1 --2- 172.21.15.40:0/3398590593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8e0073110 0x7fd8e00c64a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009d90 tx=0x7fd8d802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb5a5bc2e5172354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.983+0000 7fd8e49c4640 1 -- 172.21.15.40:0/3398590593 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8007740 con 0x7fd8e0073110 2024-04-25T11:38:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.983+0000 7fd8e49c4640 1 -- 172.21.15.40:0/3398590593 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d80078e0 con 0x7fd8e0073110 2024-04-25T11:38:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.983+0000 7fd8e49c4640 1 -- 172.21.15.40:0/3398590593 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8007cb0 con 0x7fd8e0073110 2024-04-25T11:38:10.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.984+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/3398590593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8e0073110 msgr2=0x7fd8e00c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.984+0000 7fd8e7c2a640 1 --2- 172.21.15.40:0/3398590593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8e0073110 0x7fd8e00c64a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009d90 tx=0x7fd8d802a6f0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.984+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/3398590593 shutdown_connections 2024-04-25T11:38:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.984+0000 7fd8e7c2a640 1 --2- 172.21.15.40:0/3398590593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8e0073110 0x7fd8e00c64a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.984+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/3398590593 >> 172.21.15.40:0/3398590593 conn(0x7fd8e00cd200 msgr2=0x7fd8e00cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.984+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/3398590593 shutdown_connections 2024-04-25T11:38:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.984+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/3398590593 wait complete. 2024-04-25T11:38:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.985+0000 7fd8e7c2a640 1 Processor -- start 2024-04-25T11:38:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.985+0000 7fd8e7c2a640 1 -- start start 2024-04-25T11:38:10.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e7c2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e0073110 0x7fd8e014ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e7c2a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8e0151a60 0x7fd8e00cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e7c2a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8e0151e40 0x7fd8e01ee830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e7c2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e00c9480 con 0x7fd8e0073110 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e7c2a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e00c9300 con 0x7fd8e0151e40 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e7c2a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e00c9600 con 0x7fd8e0151a60 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e59c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e0073110 0x7fd8e014ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e59c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e0073110 0x7fd8e014ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50208/0 (socket says 172.21.15.40:50208) 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e59c6640 1 -- 172.21.15.40:0/1071899519 learned_addr learned my addr 172.21.15.40:0/1071899519 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e61c7640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8e0151e40 0x7fd8e01ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e51c5640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8e0151a60 0x7fd8e00cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e59c6640 1 -- 172.21.15.40:0/1071899519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8e0151a60 msgr2=0x7fd8e00cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e59c6640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8e0151a60 0x7fd8e00cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e59c6640 1 -- 172.21.15.40:0/1071899519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8e0151e40 msgr2=0x7fd8e01ee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e59c6640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8e0151e40 0x7fd8e01ee830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e59c6640 1 -- 172.21.15.40:0/1071899519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e00cd030 con 0x7fd8e0073110 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.986+0000 7fd8e51c5640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8e0151a60 0x7fd8e00cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.987+0000 7fd8e59c6640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e0073110 0x7fd8e014ea20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c4d0 tx=0x7fd8dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.987+0000 7fd8d67fc640 1 -- 172.21.15.40:0/1071899519 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0052d0 con 0x7fd8e0073110 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.987+0000 7fd8d67fc640 1 -- 172.21.15.40:0/1071899519 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc005470 con 0x7fd8e0073110 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.987+0000 7fd8d67fc640 1 -- 172.21.15.40:0/1071899519 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc012830 con 0x7fd8e0073110 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.987+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/1071899519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e01e4980 con 0x7fd8e0073110 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.987+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/1071899519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e00ccd50 con 0x7fd8e0073110 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.987+0000 7fd8d67fc640 1 -- 172.21.15.40:0/1071899519 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd8dc0129d0 con 0x7fd8e0073110 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.987+0000 7fd8d67fc640 1 -- 172.21.15.40:0/1071899519 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd8dc012cc0 con 0x7fd8e0073110 2024-04-25T11:38:10.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.988+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/1071899519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e004fdd0 con 0x7fd8e0073110 2024-04-25T11:38:10.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:10.990+0000 7fd8d67fc640 1 -- 172.21.15.40:0/1071899519 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd8e004fdd0 con 0x7fd8e0073110 2024-04-25T11:38:11.026 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:11.039+0000 7f5bbbf21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:38:11.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.053+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/1071899519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8e0069200 con 0x7fd8e0073110 2024-04-25T11:38:11.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.053+0000 7fd8d67fc640 1 -- 172.21.15.40:0/1071899519 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd8dc0036a0 con 0x7fd8e0073110 2024-04-25T11:38:11.041 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:11.041 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:10.819113+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.054+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/1071899519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e0073110 msgr2=0x7fd8e014ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.054+0000 7fd8e7c2a640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e0073110 0x7fd8e014ea20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c4d0 tx=0x7fd8dc00c990 comp rx=0 tx=0).stop 2024-04-25T11:38:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.054+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/1071899519 shutdown_connections 2024-04-25T11:38:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.054+0000 7fd8e7c2a640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8e0151e40 0x7fd8e01ee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.054+0000 7fd8e7c2a640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8e0151a60 0x7fd8e00cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.054+0000 7fd8e7c2a640 1 --2- 172.21.15.40:0/1071899519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8e0073110 0x7fd8e014ea20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.054+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/1071899519 >> 172.21.15.40:0/1071899519 conn(0x7fd8e00cd200 msgr2=0x7fd8e0072650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.054+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/1071899519 shutdown_connections 2024-04-25T11:38:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:11.054+0000 7fd8e7c2a640 1 -- 172.21.15.40:0/1071899519 wait complete. 2024-04-25T11:38:11.052 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T11:38:11.052 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T11:38:11.076 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:11.089+0000 7feccc419040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:38:11.132 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:11.145+0000 7feccc419040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:38:11.187 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:11.200+0000 7feccc419040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:38:11.296 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:11.309+0000 7f5bbbf21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:38:11.304 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:11.317+0000 7feccc419040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:38:11.351 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:11.364+0000 7f5bbbf21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:38:11.437 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-04-25T11:38:11.449+0000 7f6e69fcb640 -1 Falling back to public interface 2024-04-25T11:38:11.437 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-25T11:38:11.449+0000 7f23c5663640 -1 Falling back to public interface 2024-04-25T11:38:11.437 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-25T11:38:11.450+0000 7f23c4612640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-25T11:38:11.438 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-25T11:38:11.450+0000 7f23c4612640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-04-25T11:38:11.441 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-04-25T11:38:11.454+0000 7eff0bffc640 -1 Falling back to public interface 2024-04-25T11:38:11.448 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-25T11:38:11.461+0000 7f84cb41c640 -1 Falling back to public interface 2024-04-25T11:38:11.450 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-25T11:38:11.463+0000 7f1792c18640 -1 Falling back to public interface 2024-04-25T11:38:11.456 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-25T11:38:11.469+0000 7f416d812640 -1 Falling back to public interface 2024-04-25T11:38:11.457 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-25T11:38:11.470+0000 7fbc10618640 -1 Falling back to public interface 2024-04-25T11:38:11.458 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-04-25T11:38:11.471+0000 7fde7b8dd640 -1 Falling back to public interface 2024-04-25T11:38:11.459 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-25T11:38:11.472+0000 7f416c0b6640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6824/0: (98) Address already in use 2024-04-25T11:38:11.480 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:11.493+0000 7f5bbbf21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:38:11.705 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:11.718+0000 7feccc419040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:38:11.775 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:11.788+0000 7f5bbbf21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:38:11.825 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:11.837+0000 7f5bbbf21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:38:11.857 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:11.949 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.962+0000 7f727ca60640 1 Processor -- start 2024-04-25T11:38:11.949 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.962+0000 7f727ca60640 1 -- start start 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f727ca60640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7278156280 0x7f727815a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f727ca60640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7278070080 0x7f727815a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f727ca60640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7278070650 con 0x7f72780c5140 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f727ca60640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f72781598f0 con 0x7f7278070080 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f727ca60640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f7278158fc0 con 0x7f7278156280 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f7276d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7278070080 0x7f727815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f7276d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7278070080 0x7f727815a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50218/0 (socket says 172.21.15.40:50218) 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f7276d9d640 1 -- 172.21.15.40:0/858965318 learned_addr learned my addr 172.21.15.40:0/858965318 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f727659c640 1 --2- 172.21.15.40:0/858965318 >> v2:172.21.15.62:3300/0 conn(0x7f7278156280 0x7f727815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f7276d9d640 1 -- 172.21.15.40:0/858965318 >> v2:172.21.15.62:3300/0 conn(0x7f7278156280 msgr2=0x7f727815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:11.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f7276d9d640 1 --2- 172.21.15.40:0/858965318 >> v2:172.21.15.62:3300/0 conn(0x7f7278156280 0x7f727815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f7276d9d640 1 -- 172.21.15.40:0/858965318 >> v1:172.21.15.62:6789/0 conn(0x7f72780c5140 legacy=0x7f727815c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.963+0000 7f7276d9d640 1 -- 172.21.15.40:0/858965318 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72781592a0 con 0x7f7278070080 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.964+0000 7f7276d9d640 1 --2- 172.21.15.40:0/858965318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 0x7f727815a640 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7278068530 tx=0x7f727002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb77415c36fe2fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.964+0000 7f727559a640 1 -- 172.21.15.40:0/858965318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270036070 con 0x7f7278070080 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.964+0000 7f727559a640 1 -- 172.21.15.40:0/858965318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7270031600 con 0x7f7278070080 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.964+0000 7f727559a640 1 -- 172.21.15.40:0/858965318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72700319f0 con 0x7f7278070080 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.965+0000 7f727ca60640 1 -- 172.21.15.40:0/858965318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 msgr2=0x7f727815a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.965+0000 7f727ca60640 1 --2- 172.21.15.40:0/858965318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 0x7f727815a640 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7278068530 tx=0x7f727002a7c0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.965+0000 7f727ca60640 1 -- 172.21.15.40:0/858965318 shutdown_connections 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.965+0000 7f727ca60640 1 --2- 172.21.15.40:0/858965318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 0x7f727815a640 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.965+0000 7f727ca60640 1 --2- 172.21.15.40:0/858965318 >> v2:172.21.15.62:3300/0 conn(0x7f7278156280 0x7f727815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.965+0000 7f727ca60640 1 -- 172.21.15.40:0/858965318 >> 172.21.15.40:0/858965318 conn(0x7f727806bfd0 msgr2=0x7f7278067630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:11.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.965+0000 7f727ca60640 1 -- 172.21.15.40:0/858965318 shutdown_connections 2024-04-25T11:38:11.954 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.965+0000 7f727ca60640 1 -- 172.21.15.40:0/858965318 wait complete. 2024-04-25T11:38:11.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.966+0000 7f727ca60640 1 Processor -- start 2024-04-25T11:38:11.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.966+0000 7f727ca60640 1 -- start start 2024-04-25T11:38:11.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f727ca60640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 0x7f72780b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f727ca60640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72780c5140 0x7f72781eea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f727ca60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7278156280 0x7f72781eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f727ca60640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7278158c90 con 0x7f7278070080 2024-04-25T11:38:11.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f727ca60640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7278156660 con 0x7f7278156280 2024-04-25T11:38:11.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f727ca60640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f727815c910 con 0x7f72780c5140 2024-04-25T11:38:11.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f727659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 0x7f72780b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f727659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 0x7f72780b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50232/0 (socket says 172.21.15.40:50232) 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f727659c640 1 -- 172.21.15.40:0/3425092985 learned_addr learned my addr 172.21.15.40:0/3425092985 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f7275d9b640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72780c5140 0x7f72781eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.967+0000 7f7276d9d640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7278156280 0x7f72781eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f727659c640 1 -- 172.21.15.40:0/3425092985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72780c5140 msgr2=0x7f72781eea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f727659c640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72780c5140 0x7f72781eea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f727659c640 1 -- 172.21.15.40:0/3425092985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7278156280 msgr2=0x7f72781eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f727659c640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7278156280 0x7f72781eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f727659c640 1 -- 172.21.15.40:0/3425092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72781e59e0 con 0x7f7278070080 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f7276d9d640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7278156280 0x7f72781eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f727659c640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 0x7f72780b2280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f726800e8a0 tx=0x7f726800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f7266ffd640 1 -- 172.21.15.40:0/3425092985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268017070 con 0x7f7278070080 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f7266ffd640 1 -- 172.21.15.40:0/3425092985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726800c940 con 0x7f7278070080 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f727ca60640 1 -- 172.21.15.40:0/3425092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72780c8270 con 0x7f7278070080 2024-04-25T11:38:11.957 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f7266ffd640 1 -- 172.21.15.40:0/3425092985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726800cd30 con 0x7f7278070080 2024-04-25T11:38:11.958 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f727ca60640 1 -- 172.21.15.40:0/3425092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72781e5c80 con 0x7f7278070080 2024-04-25T11:38:11.958 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f7266ffd640 1 -- 172.21.15.40:0/3425092985 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f726801a070 con 0x7f7278070080 2024-04-25T11:38:11.958 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.968+0000 7f7266ffd640 1 -- 172.21.15.40:0/3425092985 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f726801cb20 con 0x7f7278070080 2024-04-25T11:38:11.958 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.969+0000 7f727ca60640 1 -- 172.21.15.40:0/3425092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727804fce0 con 0x7f7278070080 2024-04-25T11:38:11.958 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:11.971+0000 7f7266ffd640 1 -- 172.21.15.40:0/3425092985 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f726801c450 con 0x7f7278070080 2024-04-25T11:38:11.978 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:11.991+0000 7feccc419040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:38:11.988 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:12.001+0000 7f5bbbf21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:38:12.021 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.034+0000 7f727ca60640 1 -- 172.21.15.40:0/3425092985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7278070950 con 0x7f7278070080 2024-04-25T11:38:12.022 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.035+0000 7f7266ffd640 1 -- 172.21.15.40:0/3425092985 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f726801c630 con 0x7f7278070080 2024-04-25T11:38:12.022 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-25T11:38:12.022 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:10.819113+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:12.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.036+0000 7f727ca60640 1 -- 172.21.15.40:0/3425092985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 msgr2=0x7f72780b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:12.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.036+0000 7f727ca60640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 0x7f72780b2280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f726800e8a0 tx=0x7f726800ed60 comp rx=0 tx=0).stop 2024-04-25T11:38:12.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.036+0000 7f727ca60640 1 -- 172.21.15.40:0/3425092985 shutdown_connections 2024-04-25T11:38:12.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.036+0000 7f727ca60640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7278156280 0x7f72781eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.023 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.036+0000 7f727ca60640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72780c5140 0x7f72781eea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.024 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.036+0000 7f727ca60640 1 --2- 172.21.15.40:0/3425092985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7278070080 0x7f72780b2280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.024 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.036+0000 7f727ca60640 1 -- 172.21.15.40:0/3425092985 >> 172.21.15.40:0/3425092985 conn(0x7f727806bfd0 msgr2=0x7f72780c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:12.024 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.036+0000 7f727ca60640 1 -- 172.21.15.40:0/3425092985 shutdown_connections 2024-04-25T11:38:12.024 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:12.036+0000 7f727ca60640 1 -- 172.21.15.40:0/3425092985 wait complete. 2024-04-25T11:38:12.032 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T11:38:12.034 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:12.047+0000 7feccc419040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:38:12.133 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:12.146+0000 7f5bbbf21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:38:12.155 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:12.168+0000 7feccc419040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:38:12.185 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:12.197+0000 7f5bbbf21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:38:12.258 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:12.271+0000 7f5bbbf21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:38:12.306 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:12.319+0000 7f5bbbf21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:38:12.448 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:12.460+0000 7feccc419040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:38:12.498 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:12.509+0000 7feccc419040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:38:12.662 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:12.674+0000 7feccc419040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:38:12.806 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:12.818+0000 7feccc419040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:38:12.808 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:12.821+0000 7f5bbbf21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:38:12.858 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:12.870+0000 7feccc419040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:38:12.929 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:12.942+0000 7f5bbbf21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:38:12.931 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:12.943+0000 7feccc419040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:38:12.982 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:12.993+0000 7feccc419040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:38:13.483 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:13.495+0000 7feccc419040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:38:13.604 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-25T11:38:13.616+0000 7feccc419040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:38:13.944 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:13.956+0000 7f5b5ca01640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T11:38:13.944 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-04-25T11:38:13.957+0000 7f5b5ca01640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T11:38:14.938 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-25T11:38:14.950+0000 7f84cb41c640 -1 osd.1 0 log_to_monitors true 2024-04-25T11:38:14.943 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-25T11:38:14.956+0000 7f23c5663640 -1 osd.6 0 log_to_monitors true 2024-04-25T11:38:14.956 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-04-25T11:38:14.968+0000 7eff0bffc640 -1 osd.7 0 log_to_monitors true 2024-04-25T11:38:14.961 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-04-25T11:38:14.974+0000 7f6e69fcb640 -1 osd.4 0 log_to_monitors true 2024-04-25T11:38:14.966 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-25T11:38:14.978+0000 7f416d812640 -1 osd.0 0 log_to_monitors true 2024-04-25T11:38:14.966 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-25T11:38:14.978+0000 7fbc10618640 -1 osd.3 0 log_to_monitors true 2024-04-25T11:38:14.971 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-25T11:38:14.983+0000 7f1792c18640 -1 osd.2 0 log_to_monitors true 2024-04-25T11:38:14.977 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-04-25T11:38:14.990+0000 7fde7b8dd640 -1 osd.5 0 log_to_monitors true 2024-04-25T11:38:17.531 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-25T11:38:17.543+0000 7f84c6da0640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T11:38:17.547 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-25T11:38:17.559+0000 7f84bebc0640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:17.552 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-04-25T11:38:17.565+0000 7fde77873640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T11:38:17.573 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-04-25T11:38:17.586+0000 7fde6f693640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:17.579 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-25T11:38:17.591+0000 7f416909d640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T11:38:17.582 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-25T11:38:17.595+0000 7f23c15f9640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T11:38:17.590 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-25T11:38:17.602+0000 7f4160ebd640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:17.599 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-04-25T11:38:17.612+0000 7f23b9419640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:17.623 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-04-25T11:38:17.636+0000 7f6e66775640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T11:38:17.633 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-04-25T11:38:17.646+0000 7eff087a6640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T11:38:17.634 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-25T11:38:17.646+0000 7fbc0bf80640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T11:38:17.637 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-04-25T11:38:17.650+0000 7f6e5dd81640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:17.639 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-25T11:38:17.651+0000 7f178e580640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T11:38:17.645 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-04-25T11:38:17.658+0000 7efeffdb2640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:17.647 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-25T11:38:17.659+0000 7fbc03da0640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:17.659 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-25T11:38:17.671+0000 7f17863a0640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:18.839 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:18.944 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.955+0000 7fd655120640 1 Processor -- start 2024-04-25T11:38:18.944 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.955+0000 7fd655120640 1 -- start start 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.956+0000 7fd655120640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd650155030 0x7fd65015b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.956+0000 7fd655120640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6500cb370 0x7fd6500d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.956+0000 7fd655120640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd65015aca0 con 0x7fd6500c77f0 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.956+0000 7fd655120640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd65015b2d0 con 0x7fd650155030 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.956+0000 7fd655120640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd650155410 con 0x7fd6500cb370 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.956+0000 7fd64ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd650155030 0x7fd65015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.956+0000 7fd64ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd650155030 0x7fd65015b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56552/0 (socket says 172.21.15.40:56552) 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.956+0000 7fd64ed9d640 1 -- 172.21.15.40:0/2701838540 learned_addr learned my addr 172.21.15.40:0/2701838540 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.956+0000 7fd64f59e640 1 --2- 172.21.15.40:0/2701838540 >> v2:172.21.15.62:3300/0 conn(0x7fd6500cb370 0x7fd6500d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.957+0000 7fd64ed9d640 1 -- 172.21.15.40:0/2701838540 >> v2:172.21.15.62:3300/0 conn(0x7fd6500cb370 msgr2=0x7fd6500d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.957+0000 7fd64ed9d640 1 --2- 172.21.15.40:0/2701838540 >> v2:172.21.15.62:3300/0 conn(0x7fd6500cb370 0x7fd6500d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.946 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.957+0000 7fd64ed9d640 1 -- 172.21.15.40:0/2701838540 >> v1:172.21.15.62:6789/0 conn(0x7fd6500c77f0 legacy=0x7fd65015e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.947 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.957+0000 7fd64ed9d640 1 -- 172.21.15.40:0/2701838540 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65015ead0 con 0x7fd650155030 2024-04-25T11:38:18.947 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.957+0000 7fd64ed9d640 1 --2- 172.21.15.40:0/2701838540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd650155030 0x7fd65015b760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd6500b5c00 tx=0x7fd64002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252fc097af5bdbbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.947 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.957+0000 7fd64dd9b640 1 -- 172.21.15.40:0/2701838540 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640036070 con 0x7fd650155030 2024-04-25T11:38:18.947 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.957+0000 7fd64dd9b640 1 -- 172.21.15.40:0/2701838540 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd640031600 con 0x7fd650155030 2024-04-25T11:38:18.947 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.957+0000 7fd64dd9b640 1 -- 172.21.15.40:0/2701838540 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6400319d0 con 0x7fd650155030 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.958+0000 7fd655120640 1 -- 172.21.15.40:0/2701838540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd650155030 msgr2=0x7fd65015b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.958+0000 7fd655120640 1 --2- 172.21.15.40:0/2701838540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd650155030 0x7fd65015b760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd6500b5c00 tx=0x7fd64002a730 comp rx=0 tx=0).stop 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.958+0000 7fd655120640 1 -- 172.21.15.40:0/2701838540 shutdown_connections 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.958+0000 7fd655120640 1 --2- 172.21.15.40:0/2701838540 >> v2:172.21.15.62:3300/0 conn(0x7fd6500cb370 0x7fd6500d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.958+0000 7fd655120640 1 --2- 172.21.15.40:0/2701838540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd650155030 0x7fd65015b760 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.958+0000 7fd655120640 1 -- 172.21.15.40:0/2701838540 >> 172.21.15.40:0/2701838540 conn(0x7fd65006c880 msgr2=0x7fd65006e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.958+0000 7fd655120640 1 -- 172.21.15.40:0/2701838540 shutdown_connections 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.958+0000 7fd655120640 1 -- 172.21.15.40:0/2701838540 wait complete. 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.960+0000 7fd655120640 1 Processor -- start 2024-04-25T11:38:18.948 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.960+0000 7fd655120640 1 -- start start 2024-04-25T11:38:18.950 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.960+0000 7fd655120640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6500c77f0 0x7fd6500c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.960+0000 7fd655120640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6500cb370 0x7fd65015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.960+0000 7fd655120640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd650155030 0x7fd65015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.960+0000 7fd655120640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd65015f090 con 0x7fd6500cb370 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.960+0000 7fd655120640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd65015bca0 con 0x7fd650155030 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.960+0000 7fd655120640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd65015af60 con 0x7fd6500c77f0 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6500cb370 0x7fd65015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6500cb370 0x7fd65015c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56564/0 (socket says 172.21.15.40:56564) 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64e59c640 1 -- 172.21.15.40:0/529184894 learned_addr learned my addr 172.21.15.40:0/529184894 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64f59e640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd650155030 0x7fd65015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64ed9d640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6500c77f0 0x7fd6500c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64e59c640 1 -- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6500c77f0 msgr2=0x7fd6500c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64e59c640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6500c77f0 0x7fd6500c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64e59c640 1 -- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd650155030 msgr2=0x7fd65015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.951 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64e59c640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd650155030 0x7fd65015c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64e59c640 1 -- 172.21.15.40:0/529184894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6501f0d60 con 0x7fd6500cb370 2024-04-25T11:38:18.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd64e59c640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6500cb370 0x7fd65015c2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd64400c3c0 tx=0x7fd64400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd6377fe640 1 -- 172.21.15.40:0/529184894 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6440052d0 con 0x7fd6500cb370 2024-04-25T11:38:18.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd6377fe640 1 -- 172.21.15.40:0/529184894 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd644005490 con 0x7fd6500cb370 2024-04-25T11:38:18.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65015dbf0 con 0x7fd6500cb370 2024-04-25T11:38:18.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd6377fe640 1 -- 172.21.15.40:0/529184894 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644007260 con 0x7fd6500cb370 2024-04-25T11:38:18.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.961+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6501f12c0 con 0x7fd6500cb370 2024-04-25T11:38:18.952 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.962+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65004fce0 con 0x7fd6500cb370 2024-04-25T11:38:18.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.963+0000 7fd6377fe640 1 -- 172.21.15.40:0/529184894 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fd644007400 con 0x7fd6500cb370 2024-04-25T11:38:18.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.963+0000 7fd6377fe640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd63c06c890 0x7fd63c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.964+0000 7fd6377fe640 1 -- 172.21.15.40:0/529184894 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fd64408aaa0 con 0x7fd6500cb370 2024-04-25T11:38:18.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.964+0000 7fd64ed9d640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd63c06c890 0x7fd63c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.953 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.964+0000 7fd64ed9d640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd63c06c890 0x7fd63c06ed50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd640033000 tx=0x7fd640009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.956 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:18.968+0000 7fd6377fe640 1 -- 172.21.15.40:0/529184894 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6501f0d60 con 0x7fd6500cb370 2024-04-25T11:38:19.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.155+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6500cc9e0 con 0x7fd6500cb370 2024-04-25T11:38:19.144 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.156+0000 7fd6377fe640 1 -- 172.21.15.40:0/529184894 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7fd644058d00 con 0x7fd6500cb370 2024-04-25T11:38:19.145 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-25T11:38:19.145 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":13,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:18.548496+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd63c06c890 msgr2=0x7fd63c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd63c06c890 0x7fd63c06ed50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd640033000 tx=0x7fd640009000 comp rx=0 tx=0).stop 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6500cb370 msgr2=0x7fd65015c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6500cb370 0x7fd65015c2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd64400c3c0 tx=0x7fd64400c880 comp rx=0 tx=0).stop 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 shutdown_connections 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd63c06c890 0x7fd63c06ed50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd650155030 0x7fd65015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6500cb370 0x7fd65015c2b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 --2- 172.21.15.40:0/529184894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6500c77f0 0x7fd6500c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 >> 172.21.15.40:0/529184894 conn(0x7fd65006c880 msgr2=0x7fd6501506e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 shutdown_connections 2024-04-25T11:38:19.147 INFO:teuthology.misc.health.smithi040.stderr:2024-04-25T11:38:19.158+0000 7fd655120640 1 -- 172.21.15.40:0/529184894 wait complete. 2024-04-25T11:38:19.156 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T11:38:19.156 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T11:38:19.157 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:38:19.157 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:38:19.159 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:38:19.159 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:38:19.159 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:38:19.161 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:38:19.161 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-25T11:38:19.161 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-25T11:38:19.163 INFO:tasks.ceph.mds.e:Started 2024-04-25T11:38:19.163 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:38:19.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:38:19.166 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:38:19.166 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:38:19.166 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:38:19.168 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:38:19.169 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-25T11:38:19.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-25T11:38:19.171 INFO:tasks.ceph.mds.f:Started 2024-04-25T11:38:19.171 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T11:38:19.171 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:19.238 INFO:tasks.ceph.mds.e.smithi040.stdout:starting mds.e at 2024-04-25T11:38:19.238 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-25T11:38:19.238 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-04-25T11:38:19.256 INFO:tasks.ceph.mds.f.smithi062.stdout:starting mds.f at 2024-04-25T11:38:19.256 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-04-25T11:38:19.257 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-04-25T11:38:19.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.291+0000 7f912a0a3640 1 Processor -- start 2024-04-25T11:38:19.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.291+0000 7f912a0a3640 1 -- start start 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f912a0a3640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91240d0660 0x7f91240cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f912a0a3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f912415ae70 0x7f912415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f912a0a3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91240cc9a0 con 0x7f91240d0a40 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f912a0a3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f912415b7f0 con 0x7f912415ae70 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f912a0a3640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f912415e580 con 0x7f91240d0660 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f9123fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f912415ae70 0x7f912415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f9122ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f91240d0a40 0x7f91240cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33426/0 (socket says 172.21.15.40:33426) 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f9122ffd640 1 -- 172.21.15.40:0/2308516901 learned_addr learned my addr 172.21.15.40:0/2308516901 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f91237fe640 1 --2- 172.21.15.40:0/2308516901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91240d0660 0x7f91240cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f91227fc640 1 -- 172.21.15.40:0/2308516901 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1639969526 0 0) 0x7f91240cc9a0 con 0x7f91240d0a40 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f91227fc640 1 -- 172.21.15.40:0/2308516901 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f910c003520 con 0x7f91240d0a40 2024-04-25T11:38:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f9123fff640 1 -- 172.21.15.40:0/2308516901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91240d0660 msgr2=0x7f91240cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.292+0000 7f9123fff640 1 --2- 172.21.15.40:0/2308516901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91240d0660 0x7f91240cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.293+0000 7f9123fff640 1 -- 172.21.15.40:0/2308516901 >> v1:172.21.15.40:6789/0 conn(0x7f91240d0a40 legacy=0x7f91240cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.293+0000 7f9123fff640 1 -- 172.21.15.40:0/2308516901 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912415e860 con 0x7f912415ae70 2024-04-25T11:38:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.293+0000 7f9123fff640 1 --2- 172.21.15.40:0/2308516901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912415ae70 0x7f912415b2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f911c009d90 tx=0x7f911c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d220474141ce8576 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.293+0000 7f91227fc640 1 -- 172.21.15.40:0/2308516901 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c007740 con 0x7f912415ae70 2024-04-25T11:38:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.293+0000 7f91227fc640 1 -- 172.21.15.40:0/2308516901 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911c0078e0 con 0x7f912415ae70 2024-04-25T11:38:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.293+0000 7f91227fc640 1 -- 172.21.15.40:0/2308516901 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c007cb0 con 0x7f912415ae70 2024-04-25T11:38:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.294+0000 7f912a0a3640 1 -- 172.21.15.40:0/2308516901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912415ae70 msgr2=0x7f912415b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.294+0000 7f912a0a3640 1 --2- 172.21.15.40:0/2308516901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912415ae70 0x7f912415b2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f911c009d90 tx=0x7f911c02a660 comp rx=0 tx=0).stop 2024-04-25T11:38:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.294+0000 7f912a0a3640 1 -- 172.21.15.40:0/2308516901 shutdown_connections 2024-04-25T11:38:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.294+0000 7f912a0a3640 1 --2- 172.21.15.40:0/2308516901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f912415ae70 0x7f912415b2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.294+0000 7f912a0a3640 1 --2- 172.21.15.40:0/2308516901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91240d0660 0x7f91240cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.294+0000 7f912a0a3640 1 -- 172.21.15.40:0/2308516901 >> 172.21.15.40:0/2308516901 conn(0x7f912406c860 msgr2=0x7f912406e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.294+0000 7f912a0a3640 1 -- 172.21.15.40:0/2308516901 shutdown_connections 2024-04-25T11:38:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.294+0000 7f912a0a3640 1 -- 172.21.15.40:0/2308516901 wait complete. 2024-04-25T11:38:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.295+0000 7f912a0a3640 1 Processor -- start 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.295+0000 7f912a0a3640 1 -- start start 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f912a0a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91240d0660 0x7f912415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f912a0a3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91240d0a40 0x7f91240cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f912a0a3640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912415ae70 0x7f9124147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f912a0a3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91240cb920 con 0x7f91240d0660 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f912a0a3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f912415ee20 con 0x7f91240d0a40 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f912a0a3640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9124154c00 con 0x7f912415ae70 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f91237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91240d0660 0x7f912415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f91237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91240d0660 0x7f912415ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56660/0 (socket says 172.21.15.40:56660) 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f91237fe640 1 -- 172.21.15.40:0/3089595017 learned_addr learned my addr 172.21.15.40:0/3089595017 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f9123fff640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912415ae70 0x7f9124147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f91237fe640 1 -- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912415ae70 msgr2=0x7f9124147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f91237fe640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912415ae70 0x7f9124147290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f91237fe640 1 -- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91240d0a40 msgr2=0x7f91240cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f91237fe640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91240d0a40 0x7f91240cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.296+0000 7f91237fe640 1 -- 172.21.15.40:0/3089595017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912415d440 con 0x7f91240d0660 2024-04-25T11:38:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.297+0000 7f91237fe640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91240d0660 0x7f912415ded0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f911400e9b0 tx=0x7f911400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.297+0000 7f910bfff640 1 -- 172.21.15.40:0/3089595017 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114017070 con 0x7f91240d0660 2024-04-25T11:38:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.297+0000 7f910bfff640 1 -- 172.21.15.40:0/3089595017 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911400c9a0 con 0x7f91240d0660 2024-04-25T11:38:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.297+0000 7f910bfff640 1 -- 172.21.15.40:0/3089595017 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911400cd90 con 0x7f91240d0660 2024-04-25T11:38:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.297+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912415d6e0 con 0x7f91240d0660 2024-04-25T11:38:19.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.297+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912414c180 con 0x7f91240d0660 2024-04-25T11:38:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.298+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91240cc9a0 con 0x7f91240d0660 2024-04-25T11:38:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.298+0000 7f910bfff640 1 -- 172.21.15.40:0/3089595017 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f911401a070 con 0x7f91240d0660 2024-04-25T11:38:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.299+0000 7f910bfff640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f911006c860 0x7f911006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.299+0000 7f910bfff640 1 -- 172.21.15.40:0/3089595017 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f911408b4e0 con 0x7f91240d0660 2024-04-25T11:38:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.299+0000 7f9122ffd640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f911006c860 0x7f911006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.299+0000 7f9122ffd640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f911006c860 0x7f911006ed20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9124154380 tx=0x7f9118007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.304+0000 7f910bfff640 1 -- 172.21.15.40:0/3089595017 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91240cc9a0 con 0x7f91240d0660 2024-04-25T11:38:19.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.488+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9124069200 con 0x7f91240d0660 2024-04-25T11:38:19.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.489+0000 7f910bfff640 1 -- 172.21.15.40:0/3089595017 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7f9114059f50 con 0x7f91240d0660 2024-04-25T11:38:19.477 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:19.477 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":13,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:18.548496+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f911006c860 msgr2=0x7f911006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f911006c860 0x7f911006ed20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9124154380 tx=0x7f9118007930 comp rx=0 tx=0).stop 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91240d0660 msgr2=0x7f912415ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91240d0660 0x7f912415ded0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f911400e9b0 tx=0x7f911400ee70 comp rx=0 tx=0).stop 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 shutdown_connections 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f911006c860 0x7f911006ed20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912415ae70 0x7f9124147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91240d0a40 0x7f91240cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 --2- 172.21.15.40:0/3089595017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91240d0660 0x7f912415ded0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 >> 172.21.15.40:0/3089595017 conn(0x7f912406c860 msgr2=0x7f91240cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 shutdown_connections 2024-04-25T11:38:19.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.491+0000 7f912a0a3640 1 -- 172.21.15.40:0/3089595017 wait complete. 2024-04-25T11:38:19.490 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-25T11:38:19.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.603+0000 7f0690136640 1 Processor -- start 2024-04-25T11:38:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.603+0000 7f0690136640 1 -- start start 2024-04-25T11:38:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.604+0000 7f0690136640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06880b6210 0x7f06880c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.604+0000 7f0690136640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f068815f060 con 0x7f0688154f50 2024-04-25T11:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.604+0000 7f0690136640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06880d1040 con 0x7f06880c8690 2024-04-25T11:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.604+0000 7f0690136640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f068815eb20 con 0x7f06880b6210 2024-04-25T11:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.604+0000 7f068d6d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06880b6210 0x7f06880c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.604+0000 7f068d6d1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f06880b6210 0x7f06880c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56668/0 (socket says 172.21.15.40:56668) 2024-04-25T11:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.604+0000 7f068d6d1640 1 -- 172.21.15.40:0/2991257320 learned_addr learned my addr 172.21.15.40:0/2991257320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.604+0000 7f068d6d1640 1 -- 172.21.15.40:0/2991257320 >> v1:172.21.15.62:6789/0 conn(0x7f06880c8690 legacy=0x7f06880c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.604+0000 7f068d6d1640 1 -- 172.21.15.40:0/2991257320 >> v1:172.21.15.40:6789/0 conn(0x7f0688154f50 legacy=0x7f06880cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.605+0000 7f068d6d1640 1 -- 172.21.15.40:0/2991257320 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06880cc150 con 0x7f06880b6210 2024-04-25T11:38:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.605+0000 7f068d6d1640 1 --2- 172.21.15.40:0/2991257320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06880b6210 0x7f06880c7a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06880cc320 tx=0x7f067802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9f44e7104fd53d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.605+0000 7f068ced0640 1 -- 172.21.15.40:0/2991257320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678037070 con 0x7f06880b6210 2024-04-25T11:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.605+0000 7f068ced0640 1 -- 172.21.15.40:0/2991257320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0678032440 con 0x7f06880b6210 2024-04-25T11:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.605+0000 7f068ced0640 1 -- 172.21.15.40:0/2991257320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678032810 con 0x7f06880b6210 2024-04-25T11:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.606+0000 7f0690136640 1 -- 172.21.15.40:0/2991257320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06880b6210 msgr2=0x7f06880c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.606+0000 7f0690136640 1 --2- 172.21.15.40:0/2991257320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06880b6210 0x7f06880c7a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06880cc320 tx=0x7f067802a210 comp rx=0 tx=0).stop 2024-04-25T11:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.606+0000 7f0690136640 1 -- 172.21.15.40:0/2991257320 shutdown_connections 2024-04-25T11:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.606+0000 7f0690136640 1 --2- 172.21.15.40:0/2991257320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06880b6210 0x7f06880c7a70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.606+0000 7f0690136640 1 -- 172.21.15.40:0/2991257320 >> 172.21.15.40:0/2991257320 conn(0x7f06880d0440 msgr2=0x7f06880d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.606+0000 7f0690136640 1 -- 172.21.15.40:0/2991257320 shutdown_connections 2024-04-25T11:38:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.606+0000 7f0690136640 1 -- 172.21.15.40:0/2991257320 wait complete. 2024-04-25T11:38:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.607+0000 7f0690136640 1 Processor -- start 2024-04-25T11:38:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.607+0000 7f0690136640 1 -- start start 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f0690136640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06880c8690 0x7f0688155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f0690136640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0688154f50 0x7f068815d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f0690136640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0688157a40 0x7f068815d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f0690136640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06880cc3f0 con 0x7f0688157a40 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f0690136640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f068815ed80 con 0x7f0688154f50 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f0690136640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f06880c8170 con 0x7f06880c8690 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068e6d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0688157a40 0x7f068815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068e6d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0688157a40 0x7f068815d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56684/0 (socket says 172.21.15.40:56684) 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068e6d3640 1 -- 172.21.15.40:0/4251132312 learned_addr learned my addr 172.21.15.40:0/4251132312 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068d6d1640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0688154f50 0x7f068815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068ded2640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06880c8690 0x7f0688155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068e6d3640 1 -- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06880c8690 msgr2=0x7f0688155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068e6d3640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06880c8690 0x7f0688155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068e6d3640 1 -- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0688154f50 msgr2=0x7f068815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068e6d3640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0688154f50 0x7f068815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.608+0000 7f068e6d3640 1 -- 172.21.15.40:0/4251132312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06881f4680 con 0x7f0688157a40 2024-04-25T11:38:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.609+0000 7f068e6d3640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0688157a40 0x7f068815d740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f068000c410 tx=0x7f068000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.610+0000 7f067e7fc640 1 -- 172.21.15.40:0/4251132312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680017070 con 0x7f0688157a40 2024-04-25T11:38:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.610+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06881f4940 con 0x7f0688157a40 2024-04-25T11:38:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.610+0000 7f067e7fc640 1 -- 172.21.15.40:0/4251132312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0680005230 con 0x7f0688157a40 2024-04-25T11:38:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.610+0000 7f067e7fc640 1 -- 172.21.15.40:0/4251132312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680011590 con 0x7f0688157a40 2024-04-25T11:38:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.610+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06881f0bf0 con 0x7f0688157a40 2024-04-25T11:38:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.611+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0688068bb0 con 0x7f0688157a40 2024-04-25T11:38:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.611+0000 7f067e7fc640 1 -- 172.21.15.40:0/4251132312 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f068001a070 con 0x7f0688157a40 2024-04-25T11:38:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.612+0000 7f067e7fc640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f066c06c890 0x7f066c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.612+0000 7f067e7fc640 1 -- 172.21.15.40:0/4251132312 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f068008a620 con 0x7f0688157a40 2024-04-25T11:38:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.612+0000 7f068ded2640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f066c06c890 0x7f066c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.612+0000 7f068ded2640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f066c06c890 0x7f066c06ed50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0684001f10 tx=0x7f0684009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.616+0000 7f067e7fc640 1 -- 172.21.15.40:0/4251132312 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06800589b0 con 0x7f0688157a40 2024-04-25T11:38:19.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.842+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06880cbe00 con 0x7f0688157a40 2024-04-25T11:38:19.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.842+0000 7f067e7fc640 1 -- 172.21.15.40:0/4251132312 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f068005ccb0 con 0x7f0688157a40 2024-04-25T11:38:19.831 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:19.831 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/4104143657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4104143657},{"type":"v1","addr":"172.21.15.40:6835","nonce":4104143657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T11:38:19.831 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 2 2024-04-25T11:38:19.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f066c06c890 msgr2=0x7f066c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f066c06c890 0x7f066c06ed50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0684001f10 tx=0x7f0684009040 comp rx=0 tx=0).stop 2024-04-25T11:38:19.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0688157a40 msgr2=0x7f068815d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0688157a40 0x7f068815d740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f068000c410 tx=0x7f068000c8d0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 shutdown_connections 2024-04-25T11:38:19.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f066c06c890 0x7f066c06ed50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0688157a40 0x7f068815d740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0688154f50 0x7f068815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 --2- 172.21.15.40:0/4251132312 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06880c8690 0x7f0688155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 >> 172.21.15.40:0/4251132312 conn(0x7f06880d0440 msgr2=0x7f0688153ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 shutdown_connections 2024-04-25T11:38:19.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:19.845+0000 7f0690136640 1 -- 172.21.15.40:0/4251132312 wait complete. 2024-04-25T11:38:21.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.959+0000 7f1a0d613640 1 Processor -- start 2024-04-25T11:38:21.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.959+0000 7f1a0d613640 1 -- start start 2024-04-25T11:38:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.960+0000 7f1a0d613640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a080c51c0 0x7f1a080c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.960+0000 7f1a0d613640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a080c9640 con 0x7f1a0806fee0 2024-04-25T11:38:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.960+0000 7f1a0d613640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a080ca2e0 con 0x7f1a08156130 2024-04-25T11:38:21.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.960+0000 7f1a0d613640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f1a080ca5c0 con 0x7f1a080c51c0 2024-04-25T11:38:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.960+0000 7f1a077fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1a0806fee0 0x7f1a08163420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33448/0 (socket says 172.21.15.40:33448) 2024-04-25T11:38:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.960+0000 7f1a077fe640 1 -- 172.21.15.40:0/2827773649 learned_addr learned my addr 172.21.15.40:0/2827773649 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.960+0000 7f1a067fc640 1 --2- 172.21.15.40:0/2827773649 >> v2:172.21.15.62:3300/0 conn(0x7f1a080c51c0 0x7f1a080c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.961+0000 7f1a05ffb640 1 -- 172.21.15.40:0/2827773649 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260253354 0 0) 0x7f1a080c9640 con 0x7f1a0806fee0 2024-04-25T11:38:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.961+0000 7f1a05ffb640 1 -- 172.21.15.40:0/2827773649 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19f0003520 con 0x7f1a0806fee0 2024-04-25T11:38:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.961+0000 7f1a067fc640 1 -- 172.21.15.40:0/2827773649 >> v1:172.21.15.62:6789/0 conn(0x7f1a08156130 legacy=0x7f1a0815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.961+0000 7f1a067fc640 1 -- 172.21.15.40:0/2827773649 >> v1:172.21.15.40:6789/0 conn(0x7f1a0806fee0 legacy=0x7f1a08163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.961+0000 7f1a067fc640 1 -- 172.21.15.40:0/2827773649 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a080ca8a0 con 0x7f1a080c51c0 2024-04-25T11:38:21.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.961+0000 7f1a067fc640 1 --2- 172.21.15.40:0/2827773649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a080c51c0 0x7f1a080c8d50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0097a0 tx=0x7f19fc02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0432fcbd4e4fd8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:21.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a05ffb640 1 -- 172.21.15.40:0/2827773649 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc037070 con 0x7f1a080c51c0 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a05ffb640 1 -- 172.21.15.40:0/2827773649 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc032640 con 0x7f1a080c51c0 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a05ffb640 1 -- 172.21.15.40:0/2827773649 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc032a10 con 0x7f1a080c51c0 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a0d613640 1 -- 172.21.15.40:0/2827773649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a080c51c0 msgr2=0x7f1a080c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a0d613640 1 --2- 172.21.15.40:0/2827773649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a080c51c0 0x7f1a080c8d50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0097a0 tx=0x7f19fc02a2c0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a0d613640 1 -- 172.21.15.40:0/2827773649 shutdown_connections 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a0d613640 1 --2- 172.21.15.40:0/2827773649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a080c51c0 0x7f1a080c8d50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a0d613640 1 -- 172.21.15.40:0/2827773649 >> 172.21.15.40:0/2827773649 conn(0x7f1a0806bd90 msgr2=0x7f1a080676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a0d613640 1 -- 172.21.15.40:0/2827773649 shutdown_connections 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.962+0000 7f1a0d613640 1 -- 172.21.15.40:0/2827773649 wait complete. 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.963+0000 7f1a0d613640 1 Processor -- start 2024-04-25T11:38:21.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.963+0000 7f1a0d613640 1 -- start start 2024-04-25T11:38:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a0d613640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0806fee0 0x7f1a0815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a0d613640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a080c51c0 0x7f1a08156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a0d613640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a08156130 0x7f1a0815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a0d613640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1a080cbb90 con 0x7f1a080c51c0 2024-04-25T11:38:21.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a0d613640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a080ce640 con 0x7f1a0806fee0 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a0d613640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1a080c9290 con 0x7f1a08156130 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a067fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a080c51c0 0x7f1a08156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a067fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a080c51c0 0x7f1a08156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56694/0 (socket says 172.21.15.40:56694) 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a067fc640 1 -- 172.21.15.40:0/1746312167 learned_addr learned my addr 172.21.15.40:0/1746312167 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a06ffd640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0806fee0 0x7f1a0815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.964+0000 7f1a077fe640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a08156130 0x7f1a0815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f1a067fc640 1 -- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a08156130 msgr2=0x7f1a0815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f1a067fc640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a08156130 0x7f1a0815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f1a067fc640 1 -- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0806fee0 msgr2=0x7f1a0815c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f1a067fc640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0806fee0 0x7f1a0815c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f1a067fc640 1 -- 172.21.15.40:0/1746312167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a081f1a30 con 0x7f1a080c51c0 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f1a067fc640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a080c51c0 0x7f1a08156780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0097a0 tx=0x7f19fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f19ef7fe640 1 -- 172.21.15.40:0/1746312167 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc037050 con 0x7f1a080c51c0 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f19ef7fe640 1 -- 172.21.15.40:0/1746312167 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc038040 con 0x7f1a080c51c0 2024-04-25T11:38:21.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a081f1d10 con 0x7f1a080c51c0 2024-04-25T11:38:21.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f19ef7fe640 1 -- 172.21.15.40:0/1746312167 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc004270 con 0x7f1a080c51c0 2024-04-25T11:38:21.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.965+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a081f22d0 con 0x7f1a080c51c0 2024-04-25T11:38:21.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.966+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a08069200 con 0x7f1a080c51c0 2024-04-25T11:38:21.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.967+0000 7f19ef7fe640 1 -- 172.21.15.40:0/1746312167 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f19fc030900 con 0x7f1a080c51c0 2024-04-25T11:38:21.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.967+0000 7f19ef7fe640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f19f406c860 0x7f19f406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.967+0000 7f19ef7fe640 1 -- 172.21.15.40:0/1746312167 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f19fc0ac1d0 con 0x7f1a080c51c0 2024-04-25T11:38:21.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.967+0000 7f1a06ffd640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f19f406c860 0x7f19f406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.968+0000 7f1a06ffd640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f19f406c860 0x7f19f406ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a00001e70 tx=0x7f1a00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:21.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:21.972+0000 7f19ef7fe640 1 -- 172.21.15.40:0/1746312167 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19fc07a470 con 0x7f1a080c51c0 2024-04-25T11:38:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.194+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a08070a50 con 0x7f1a080c51c0 2024-04-25T11:38:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.195+0000 7f19ef7fe640 1 -- 172.21.15.40:0/1746312167 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5509 (secure 0 0 0) 0x7f19fc07e770 con 0x7f1a080c51c0 2024-04-25T11:38:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:22.183 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1073487120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1073487120},{"type":"v1","addr":"172.21.15.40:6837","nonce":1073487120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6838/3333589902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3333589902},{"type":"v1","addr":"172.21.15.62:6838","nonce":3333589902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/4104143657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4104143657},{"type":"v1","addr":"172.21.15.40:6835","nonce":4104143657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6836/535060457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":535060457},{"type":"v1","addr":"172.21.15.40:6836","nonce":535060457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4294,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3758619338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3758619338},{"type":"v1","addr":"172.21.15.62:6835","nonce":3758619338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/2121129816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6837","nonce":2121129816},{"type":"v1","addr":"172.21.15.62:6839","nonce":2121129816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:38:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2024-04-25T11:38:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f19f406c860 msgr2=0x7f19f406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f19f406c860 0x7f19f406ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a00001e70 tx=0x7f1a00009040 comp rx=0 tx=0).stop 2024-04-25T11:38:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a080c51c0 msgr2=0x7f1a08156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a080c51c0 0x7f1a08156780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0097a0 tx=0x7f19fc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 shutdown_connections 2024-04-25T11:38:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f19f406c860 0x7f19f406ed20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a08156130 0x7f1a0815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a080c51c0 0x7f1a08156780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 --2- 172.21.15.40:0/1746312167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0806fee0 0x7f1a0815c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 >> 172.21.15.40:0/1746312167 conn(0x7f1a0806bd90 msgr2=0x7f1a080711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 shutdown_connections 2024-04-25T11:38:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.197+0000 7f1a0d613640 1 -- 172.21.15.40:0/1746312167 wait complete. 2024-04-25T11:38:22.196 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.308+0000 7fe4dede2640 1 Processor -- start 2024-04-25T11:38:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.308+0000 7fe4dede2640 1 -- start start 2024-04-25T11:38:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dede2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4d80cccf0 0x7fe4d80cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dede2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d80d0c00 con 0x7fe4d8155140 2024-04-25T11:38:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dede2640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d80cd660 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dede2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d815e200 con 0x7fe4d80cccf0 2024-04-25T11:38:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dcb7e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe4d8155140 0x7fe4d80d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33462/0 (socket says 172.21.15.40:33462) 2024-04-25T11:38:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dd37f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4d80cccf0 0x7fe4d80cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dcb7e640 1 -- 172.21.15.40:0/945551324 learned_addr learned my addr 172.21.15.40:0/945551324 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dd37f640 1 -- 172.21.15.40:0/945551324 >> v1:172.21.15.62:6789/0 conn(0x7fe4d80c7ab0 legacy=0x7fe4d80cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dd37f640 1 -- 172.21.15.40:0/945551324 >> v1:172.21.15.40:6789/0 conn(0x7fe4d8155140 legacy=0x7fe4d80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.309+0000 7fe4dd37f640 1 -- 172.21.15.40:0/945551324 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d815e4e0 con 0x7fe4d80cccf0 2024-04-25T11:38:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.310+0000 7fe4dd37f640 1 --2- 172.21.15.40:0/945551324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80cccf0 0x7fe4d80cd120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009d90 tx=0x7fe4d002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26db7ed38d97df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.310+0000 7fe4cf7fe640 1 -- 172.21.15.40:0/945551324 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0007740 con 0x7fe4d80cccf0 2024-04-25T11:38:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.310+0000 7fe4cf7fe640 1 -- 172.21.15.40:0/945551324 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d00078e0 con 0x7fe4d80cccf0 2024-04-25T11:38:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.310+0000 7fe4cf7fe640 1 -- 172.21.15.40:0/945551324 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0007cb0 con 0x7fe4d80cccf0 2024-04-25T11:38:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.310+0000 7fe4dede2640 1 -- 172.21.15.40:0/945551324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80cccf0 msgr2=0x7fe4d80cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.310+0000 7fe4dede2640 1 --2- 172.21.15.40:0/945551324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80cccf0 0x7fe4d80cd120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009d90 tx=0x7fe4d002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.310+0000 7fe4dede2640 1 -- 172.21.15.40:0/945551324 shutdown_connections 2024-04-25T11:38:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.310+0000 7fe4dede2640 1 --2- 172.21.15.40:0/945551324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80cccf0 0x7fe4d80cd120 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.310+0000 7fe4dede2640 1 -- 172.21.15.40:0/945551324 >> 172.21.15.40:0/945551324 conn(0x7fe4d806e320 msgr2=0x7fe4d806c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.311+0000 7fe4dede2640 1 -- 172.21.15.40:0/945551324 shutdown_connections 2024-04-25T11:38:22.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.311+0000 7fe4dede2640 1 -- 172.21.15.40:0/945551324 wait complete. 2024-04-25T11:38:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.312+0000 7fe4dede2640 1 Processor -- start 2024-04-25T11:38:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.312+0000 7fe4dede2640 1 -- start start 2024-04-25T11:38:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.312+0000 7fe4dede2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80c7ab0 0x7fe4d80c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.312+0000 7fe4dede2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4d80cccf0 0x7fe4d81e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dede2640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4d8155140 0x7fe4d80c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dede2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d815ed80 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dede2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d815eaa0 con 0x7fe4d80cccf0 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dede2640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d815f060 con 0x7fe4d8155140 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dcb7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80c7ab0 0x7fe4d80c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dcb7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80c7ab0 0x7fe4d80c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56714/0 (socket says 172.21.15.40:56714) 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dcb7e640 1 -- 172.21.15.40:0/669477202 learned_addr learned my addr 172.21.15.40:0/669477202 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4cffff640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4d80cccf0 0x7fe4d81e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dd37f640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4d8155140 0x7fe4d80c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dcb7e640 1 -- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4d8155140 msgr2=0x7fe4d80c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dcb7e640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4d8155140 0x7fe4d80c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dcb7e640 1 -- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4d80cccf0 msgr2=0x7fe4d81e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dcb7e640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4d80cccf0 0x7fe4d81e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dcb7e640 1 -- 172.21.15.40:0/669477202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d81f0cd0 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dcb7e640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80c7ab0 0x7fe4d80c6af0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400c4a0 tx=0x7fe4d400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4cd7fa640 1 -- 172.21.15.40:0/669477202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4017070 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4cd7fa640 1 -- 172.21.15.40:0/669477202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d4005ce0 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d81f0fb0 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4cd7fa640 1 -- 172.21.15.40:0/669477202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4007270 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.313+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d81f1570 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.314+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d80caae0 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.315+0000 7fe4cd7fa640 1 -- 172.21.15.40:0/669477202 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fe4d401a070 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.315+0000 7fe4cd7fa640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe4c406c860 0x7fe4c406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.315+0000 7fe4cd7fa640 1 -- 172.21.15.40:0/669477202 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe4d408a9d0 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.315+0000 7fe4cffff640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe4c406c860 0x7fe4c406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.316+0000 7fe4cffff640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe4c406c860 0x7fe4c406ed20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe4d81e82d0 tx=0x7fe4c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.319+0000 7fe4cd7fa640 1 -- 172.21.15.40:0/669477202 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4d80caae0 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.504+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4d8068bb0 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.505+0000 7fe4cd7fa640 1 -- 172.21.15.40:0/669477202 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7fe4d4058b50 con 0x7fe4d80c7ab0 2024-04-25T11:38:22.493 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:22.493 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:21.625809+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe4c406c860 msgr2=0x7fe4c406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe4c406c860 0x7fe4c406ed20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe4d81e82d0 tx=0x7fe4c8009040 comp rx=0 tx=0).stop 2024-04-25T11:38:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80c7ab0 msgr2=0x7fe4d80c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80c7ab0 0x7fe4d80c6af0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400c4a0 tx=0x7fe4d400c960 comp rx=0 tx=0).stop 2024-04-25T11:38:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 shutdown_connections 2024-04-25T11:38:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe4c406c860 0x7fe4c406ed20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4d8155140 0x7fe4d80c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4d80cccf0 0x7fe4d81e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 --2- 172.21.15.40:0/669477202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80c7ab0 0x7fe4d80c6af0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 >> 172.21.15.40:0/669477202 conn(0x7fe4d806e320 msgr2=0x7fe4d80cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 shutdown_connections 2024-04-25T11:38:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.507+0000 7fe4dede2640 1 -- 172.21.15.40:0/669477202 wait complete. 2024-04-25T11:38:22.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:38:18.972986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:38:22.506 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:38:22.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.621+0000 7fb24ecfa640 1 Processor -- start 2024-04-25T11:38:22.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.622+0000 7fb24ecfa640 1 -- start start 2024-04-25T11:38:22.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.622+0000 7fb24ecfa640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248152f10 0x7fb24815c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.622+0000 7fb24ecfa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb248158ad0 0x7fb248163450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.622+0000 7fb24ecfa640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2480c9490 con 0x7fb24815cd70 2024-04-25T11:38:22.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.622+0000 7fb24ecfa640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb24815c0c0 con 0x7fb248158ad0 2024-04-25T11:38:22.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.622+0000 7fb24ecfa640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb248159030 con 0x7fb248152f10 2024-04-25T11:38:22.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.622+0000 7fb24d297640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb248158ad0 0x7fb248163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.623+0000 7fb24d297640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb248158ad0 0x7fb248163450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56718/0 (socket says 172.21.15.40:56718) 2024-04-25T11:38:22.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.623+0000 7fb24d297640 1 -- 172.21.15.40:0/440345942 learned_addr learned my addr 172.21.15.40:0/440345942 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:22.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.623+0000 7fb24ca96640 1 --2- 172.21.15.40:0/440345942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248152f10 0x7fb24815c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.623+0000 7fb24d297640 1 -- 172.21.15.40:0/440345942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248152f10 msgr2=0x7fb24815c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:22.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.623+0000 7fb24d297640 1 --2- 172.21.15.40:0/440345942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248152f10 0x7fb24815c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.623+0000 7fb24d297640 1 -- 172.21.15.40:0/440345942 >> v1:172.21.15.62:6789/0 conn(0x7fb24815cd70 legacy=0x7fb2480cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.623+0000 7fb24d297640 1 -- 172.21.15.40:0/440345942 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb248159310 con 0x7fb248158ad0 2024-04-25T11:38:22.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.623+0000 7fb24d297640 1 --2- 172.21.15.40:0/440345942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248158ad0 0x7fb248163450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb240009e50 tx=0x7fb24002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e736cf2fe1e0f188 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb2477fe640 1 -- 172.21.15.40:0/440345942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240036070 con 0x7fb248158ad0 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb2477fe640 1 -- 172.21.15.40:0/440345942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb240031600 con 0x7fb248158ad0 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb2477fe640 1 -- 172.21.15.40:0/440345942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400319d0 con 0x7fb248158ad0 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb24ecfa640 1 -- 172.21.15.40:0/440345942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248158ad0 msgr2=0x7fb248163450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb24ecfa640 1 --2- 172.21.15.40:0/440345942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248158ad0 0x7fb248163450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb240009e50 tx=0x7fb24002a680 comp rx=0 tx=0).stop 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb24ecfa640 1 -- 172.21.15.40:0/440345942 shutdown_connections 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb24ecfa640 1 --2- 172.21.15.40:0/440345942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248158ad0 0x7fb248163450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb24ecfa640 1 --2- 172.21.15.40:0/440345942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248152f10 0x7fb24815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb24ecfa640 1 -- 172.21.15.40:0/440345942 >> 172.21.15.40:0/440345942 conn(0x7fb24806bfa0 msgr2=0x7fb2480676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb24ecfa640 1 -- 172.21.15.40:0/440345942 shutdown_connections 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.624+0000 7fb24ecfa640 1 -- 172.21.15.40:0/440345942 wait complete. 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.625+0000 7fb24ecfa640 1 Processor -- start 2024-04-25T11:38:22.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.625+0000 7fb24ecfa640 1 -- start start 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.626+0000 7fb24ecfa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248152f10 0x7fb24814f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.626+0000 7fb24ecfa640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248158ad0 0x7fb2481570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.626+0000 7fb24ecfa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24815cd70 0x7fb2480cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.626+0000 7fb24ecfa640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb24815c310 con 0x7fb248152f10 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.626+0000 7fb24ecfa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481598d0 con 0x7fb24815cd70 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.626+0000 7fb24ecfa640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb2480c5490 con 0x7fb248158ad0 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.626+0000 7fb24ca96640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248152f10 0x7fb24814f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.626+0000 7fb24ca96640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248152f10 0x7fb24814f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56730/0 (socket says 172.21.15.40:56730) 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.626+0000 7fb24ca96640 1 -- 172.21.15.40:0/1725956029 learned_addr learned my addr 172.21.15.40:0/1725956029 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb247fff640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248158ad0 0x7fb2481570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24d297640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24815cd70 0x7fb2480cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24ca96640 1 -- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248158ad0 msgr2=0x7fb2481570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24ca96640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248158ad0 0x7fb2481570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24ca96640 1 -- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24815cd70 msgr2=0x7fb2480cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24ca96640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24815cd70 0x7fb2480cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24ca96640 1 -- 172.21.15.40:0/1725956029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2481f41a0 con 0x7fb248152f10 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24d297640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24815cd70 0x7fb2480cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24ca96640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248152f10 0x7fb24814f260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00e9b0 tx=0x7fb23c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb2457fa640 1 -- 172.21.15.40:0/1725956029 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c017070 con 0x7fb248152f10 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb2457fa640 1 -- 172.21.15.40:0/1725956029 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23c00c9a0 con 0x7fb248152f10 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24814f8a0 con 0x7fb248152f10 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb2457fa640 1 -- 172.21.15.40:0/1725956029 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00cd90 con 0x7fb248152f10 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.627+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2481f4720 con 0x7fb248152f10 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.628+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2480682b0 con 0x7fb248152f10 2024-04-25T11:38:22.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.629+0000 7fb2457fa640 1 -- 172.21.15.40:0/1725956029 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fb23c01a070 con 0x7fb248152f10 2024-04-25T11:38:22.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.629+0000 7fb2457fa640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb22c06c860 0x7fb22c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.629+0000 7fb2457fa640 1 -- 172.21.15.40:0/1725956029 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb23c08ba40 con 0x7fb248152f10 2024-04-25T11:38:22.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.629+0000 7fb247fff640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb22c06c860 0x7fb22c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.630+0000 7fb247fff640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb22c06c860 0x7fb22c06ed20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb2480cb150 tx=0x7fb234009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.634+0000 7fb2457fa640 1 -- 172.21.15.40:0/1725956029 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2480682b0 con 0x7fb248152f10 2024-04-25T11:38:22.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.823+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2480cbd00 con 0x7fb248152f10 2024-04-25T11:38:22.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.824+0000 7fb2457fa640 1 -- 172.21.15.40:0/1725956029 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fb23c059c50 con 0x7fb248152f10 2024-04-25T11:38:22.812 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-25T11:38:22.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb22c06c860 msgr2=0x7fb22c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb22c06c860 0x7fb22c06ed20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb2480cb150 tx=0x7fb234009040 comp rx=0 tx=0).stop 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248152f10 msgr2=0x7fb24814f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248152f10 0x7fb24814f260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00e9b0 tx=0x7fb23c00ee70 comp rx=0 tx=0).stop 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 shutdown_connections 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb22c06c860 0x7fb22c06ed20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24815cd70 0x7fb2480cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb248158ad0 0x7fb2481570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 --2- 172.21.15.40:0/1725956029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb248152f10 0x7fb24814f260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 >> 172.21.15.40:0/1725956029 conn(0x7fb24806bfa0 msgr2=0x7fb2480c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 shutdown_connections 2024-04-25T11:38:22.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.826+0000 7fb24ecfa640 1 -- 172.21.15.40:0/1725956029 wait complete. 2024-04-25T11:38:22.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:38:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.937+0000 7ff9e2e2a640 1 Processor -- start 2024-04-25T11:38:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.937+0000 7ff9e2e2a640 1 -- start start 2024-04-25T11:38:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e2e2a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9dc152c40 0x7ff9dc0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e2e2a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff9dc0c74c0 0x7ff9dc0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e2e2a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 0x7ff9dc0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e2e2a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc153020 con 0x7ff9dc152c40 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e2e2a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc070ea0 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e2e2a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc06bfa0 con 0x7ff9dc0c7d80 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e0bc6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9dc152c40 0x7ff9dc0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e0bc6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9dc152c40 0x7ff9dc0c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57256/0 (socket says 172.21.15.40:57256) 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e0bc6640 1 -- 172.21.15.40:0/3792666798 learned_addr learned my addr 172.21.15.40:0/3792666798 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9dbfff640 1 --2- 172.21.15.40:0/3792666798 >> v2:172.21.15.62:3300/0 conn(0x7ff9dc0c74c0 0x7ff9dc0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e13c7640 1 --2- 172.21.15.40:0/3792666798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 0x7ff9dc0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e0bc6640 1 -- 172.21.15.40:0/3792666798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 msgr2=0x7ff9dc0ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e0bc6640 1 --2- 172.21.15.40:0/3792666798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 0x7ff9dc0ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e0bc6640 1 -- 172.21.15.40:0/3792666798 >> v2:172.21.15.62:3300/0 conn(0x7ff9dc0c74c0 msgr2=0x7ff9dc0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e0bc6640 1 --2- 172.21.15.40:0/3792666798 >> v2:172.21.15.62:3300/0 conn(0x7ff9dc0c74c0 0x7ff9dc0ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e0bc6640 1 -- 172.21.15.40:0/3792666798 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc06c280 con 0x7ff9dc152c40 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9e13c7640 1 --2- 172.21.15.40:0/3792666798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 0x7ff9dc0ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.938+0000 7ff9dbfff640 1 --2- 172.21.15.40:0/3792666798 >> v2:172.21.15.62:3300/0 conn(0x7ff9dc0c74c0 0x7ff9dc0ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.939+0000 7ff9e0bc6640 1 --2- 172.21.15.40:0/3792666798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc152c40 0x7ff9dc0c98c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc06dec0 tx=0x7ff9d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78d97ef23b09544 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.939+0000 7ff9db7fe640 1 -- 172.21.15.40:0/3792666798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0037070 con 0x7ff9dc152c40 2024-04-25T11:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.939+0000 7ff9db7fe640 1 -- 172.21.15.40:0/3792666798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d0032440 con 0x7ff9dc152c40 2024-04-25T11:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.939+0000 7ff9db7fe640 1 -- 172.21.15.40:0/3792666798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0032810 con 0x7ff9dc152c40 2024-04-25T11:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.940+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3792666798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc152c40 msgr2=0x7ff9dc0c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.940+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3792666798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc152c40 0x7ff9dc0c98c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc06dec0 tx=0x7ff9d002a270 comp rx=0 tx=0).stop 2024-04-25T11:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.940+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3792666798 shutdown_connections 2024-04-25T11:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.940+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3792666798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 0x7ff9dc0ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.940+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3792666798 >> v2:172.21.15.62:3300/0 conn(0x7ff9dc0c74c0 0x7ff9dc0ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.940+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3792666798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc152c40 0x7ff9dc0c98c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.940+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3792666798 >> 172.21.15.40:0/3792666798 conn(0x7ff9dc0727a0 msgr2=0x7ff9dc072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.940+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3792666798 shutdown_connections 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.940+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3792666798 wait complete. 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.941+0000 7ff9e2e2a640 1 Processor -- start 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.941+0000 7ff9e2e2a640 1 -- start start 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e2e2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc0c74c0 0x7ff9dc0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e2e2a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 0x7ff9dc1438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e2e2a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9dc152c40 0x7ff9dc143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e2e2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc070b30 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e2e2a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc070850 con 0x7ff9dc152c40 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e2e2a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc06bc60 con 0x7ff9dc0c7d80 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e0bc6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc0c74c0 0x7ff9dc0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e0bc6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc0c74c0 0x7ff9dc0cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57262/0 (socket says 172.21.15.40:57262) 2024-04-25T11:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e0bc6640 1 -- 172.21.15.40:0/3991686154 learned_addr learned my addr 172.21.15.40:0/3991686154 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9dbfff640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 0x7ff9dc1438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e13c7640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9dc152c40 0x7ff9dc143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e0bc6640 1 -- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 msgr2=0x7ff9dc1438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e0bc6640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 0x7ff9dc1438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e0bc6640 1 -- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9dc152c40 msgr2=0x7ff9dc143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e0bc6640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9dc152c40 0x7ff9dc143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e0bc6640 1 -- 172.21.15.40:0/3991686154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0c57f0 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e13c7640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9dc152c40 0x7ff9dc143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e0bc6640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc0c74c0 0x7ff9dc0cd6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9d002a2b0 tx=0x7ff9d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9d97fa640 1 -- 172.21.15.40:0/3991686154 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0037040 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9d97fa640 1 -- 172.21.15.40:0/3991686154 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d0042070 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.942+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc15ba20 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.943+0000 7ff9d97fa640 1 -- 172.21.15.40:0/3991686154 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0030bb0 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.943+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc15bfa0 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.944+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc068bb0 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.944+0000 7ff9d97fa640 1 -- 172.21.15.40:0/3991686154 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ff9d0047460 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.944+0000 7ff9d97fa640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff9c006c860 0x7ff9c006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.944+0000 7ff9d97fa640 1 -- 172.21.15.40:0/3991686154 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff9d00ac2d0 con 0x7ff9dc0c74c0 2024-04-25T11:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.944+0000 7ff9dbfff640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff9c006c860 0x7ff9c006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.945+0000 7ff9dbfff640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff9c006c860 0x7ff9c006ed20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006fd0 tx=0x7ff9c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:22.949+0000 7ff9d97fa640 1 -- 172.21.15.40:0/3991686154 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9d0045360 con 0x7ff9dc0c74c0 2024-04-25T11:38:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.133+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff9dc06ba40 con 0x7ff9dc0c74c0 2024-04-25T11:38:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.133+0000 7ff9d97fa640 1 -- 172.21.15.40:0/3991686154 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7ff9d007abc0 con 0x7ff9dc0c74c0 2024-04-25T11:38:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:23.122 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-25T11:38:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-25T11:38:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-25T11:38:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-25T11:38:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T11:38:23.122 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff9c006c860 msgr2=0x7ff9c006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff9c006c860 0x7ff9c006ed20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006fd0 tx=0x7ff9c8008040 comp rx=0 tx=0).stop 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc0c74c0 msgr2=0x7ff9dc0cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc0c74c0 0x7ff9dc0cd6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9d002a2b0 tx=0x7ff9d00046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 shutdown_connections 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff9c006c860 0x7ff9c006ed20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9dc152c40 0x7ff9dc143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9dc0c7d80 0x7ff9dc1438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 --2- 172.21.15.40:0/3991686154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9dc0c74c0 0x7ff9dc0cd6c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 >> 172.21.15.40:0/3991686154 conn(0x7ff9dc0727a0 msgr2=0x7ff9dc151440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.135+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 shutdown_connections 2024-04-25T11:38:23.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.136+0000 7ff9e2e2a640 1 -- 172.21.15.40:0/3991686154 wait complete. 2024-04-25T11:38:23.134 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:38:23.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:38:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.251+0000 7f5dadca9640 1 Processor -- start 2024-04-25T11:38:23.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.251+0000 7f5dadca9640 1 -- start start 2024-04-25T11:38:23.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.252+0000 7f5dadca9640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155300 0x7f5da80cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.252+0000 7f5dadca9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da815b5e0 con 0x7f5da8154f20 2024-04-25T11:38:23.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.252+0000 7f5dadca9640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da815b8c0 con 0x7f5da80cd630 2024-04-25T11:38:23.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.252+0000 7f5dadca9640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5da80ccbe0 con 0x7f5da8155300 2024-04-25T11:38:23.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.252+0000 7f5da77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5da8154f20 0x7f5da815ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43834/0 (socket says 172.21.15.40:43834) 2024-04-25T11:38:23.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.252+0000 7f5da77fe640 1 -- 172.21.15.40:0/3790244989 learned_addr learned my addr 172.21.15.40:0/3790244989 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:23.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.252+0000 7f5da6ffd640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155300 0x7f5da80cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.252+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3327082327 0 0) 0x7f5da815b5e0 con 0x7f5da8154f20 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.252+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d90003520 con 0x7f5da8154f20 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3782986242 0 0) 0x7f5d90003520 con 0x7f5da8154f20 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155300 msgr2=0x7f5da80cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155300 0x7f5da80cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 >> v1:172.21.15.62:6789/0 conn(0x7f5da80cd630 legacy=0x7f5da80cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da815bba0 con 0x7f5da8154f20 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7f5d98001670 con 0x7f5da8154f20 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 >> v1:172.21.15.40:6789/0 conn(0x7f5da8154f20 legacy=0x7f5da815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d900039e0 0x7f5d90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d90004800 0x7f5d90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155300 0x7f5d900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80ccbe0 con 0x7f5d90004800 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9000ba70 con 0x7f5d900039e0 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9000bc40 con 0x7f5da8155300 2024-04-25T11:38:23.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da6ffd640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d90004800 0x7f5d90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da77fe640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d900039e0 0x7f5d90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da7fff640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155300 0x7f5d900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da6ffd640 1 -- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155300 msgr2=0x7f5d900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da6ffd640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155300 0x7f5d900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da6ffd640 1 -- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d900039e0 msgr2=0x7f5d90003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da6ffd640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d900039e0 0x7f5d90003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da6ffd640 1 -- 172.21.15.40:0/3790244989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d90003520 con 0x7f5d90004800 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.253+0000 7f5da7fff640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155300 0x7f5d900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.254+0000 7f5da6ffd640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d90004800 0x7f5d90004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009a90 tx=0x7f5d90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112340a7fe46fbcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.254+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c036070 con 0x7f5d90004800 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.254+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9c031400 con 0x7f5d90004800 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.254+0000 7f5da67fc640 1 -- 172.21.15.40:0/3790244989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c009690 con 0x7f5d90004800 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.254+0000 7f5dadca9640 1 -- 172.21.15.40:0/3790244989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d90004800 msgr2=0x7f5d90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.254+0000 7f5dadca9640 1 --2- 172.21.15.40:0/3790244989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d90004800 0x7f5d90004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009a90 tx=0x7f5d90005830 comp rx=0 tx=0).stop 2024-04-25T11:38:23.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.254+0000 7f5da7fff640 1 -- 172.21.15.40:0/3790244989 reap_dead start 2024-04-25T11:38:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.255+0000 7f5dadca9640 1 -- 172.21.15.40:0/3790244989 shutdown_connections 2024-04-25T11:38:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.255+0000 7f5dadca9640 1 -- 172.21.15.40:0/3790244989 >> 172.21.15.40:0/3790244989 conn(0x7f5da806c9a0 msgr2=0x7f5da806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.255+0000 7f5dadca9640 1 -- 172.21.15.40:0/3790244989 shutdown_connections 2024-04-25T11:38:23.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.255+0000 7f5dadca9640 1 -- 172.21.15.40:0/3790244989 wait complete. 2024-04-25T11:38:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.256+0000 7f5dadca9640 1 Processor -- start 2024-04-25T11:38:23.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.256+0000 7f5dadca9640 1 -- start start 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5dadca9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90004800 0x7f5da81f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5dadca9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da8156be0 0x7f5da80ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5dadca9640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155710 0x7f5da8157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5dadca9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80c78a0 con 0x7f5da8156be0 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5dadca9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80cd1a0 con 0x7f5d90004800 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5dadca9640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5da80cc7b0 con 0x7f5da8155710 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da8156be0 0x7f5da80ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da8156be0 0x7f5da80ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57290/0 (socket says 172.21.15.40:57290) 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da6ffd640 1 -- 172.21.15.40:0/94900112 learned_addr learned my addr 172.21.15.40:0/94900112 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da7fff640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155710 0x7f5da8157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da77fe640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90004800 0x7f5da81f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da6ffd640 1 -- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155710 msgr2=0x7f5da8157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da6ffd640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155710 0x7f5da8157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da6ffd640 1 -- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90004800 msgr2=0x7f5da81f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da6ffd640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90004800 0x7f5da81f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da6ffd640 1 -- 172.21.15.40:0/94900112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da81580b0 con 0x7f5da8156be0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5da6ffd640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da8156be0 0x7f5da80ce600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009a90 tx=0x7f5d9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5d8ffff640 1 -- 172.21.15.40:0/94900112 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c041020 con 0x7f5da8156be0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5d8ffff640 1 -- 172.21.15.40:0/94900112 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9c03c470 con 0x7f5da8156be0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da8158350 con 0x7f5da8156be0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5d8ffff640 1 -- 172.21.15.40:0/94900112 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c036040 con 0x7f5da8156be0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.257+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da81f0840 con 0x7f5da8156be0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.258+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da8069200 con 0x7f5da8156be0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.259+0000 7f5d8ffff640 1 -- 172.21.15.40:0/94900112 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f5d9c044020 con 0x7f5da8156be0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.259+0000 7f5d8ffff640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5d9406c890 0x7f5d9406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.259+0000 7f5d8ffff640 1 -- 172.21.15.40:0/94900112 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5d9c0abf50 con 0x7f5da8156be0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.259+0000 7f5da77fe640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5d9406c890 0x7f5d9406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.260+0000 7f5da77fe640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5d9406c890 0x7f5d9406ed50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d98007c80 tx=0x7f5d98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:23.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.264+0000 7f5d8ffff640 1 -- 172.21.15.40:0/94900112 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5da8069200 con 0x7f5da8156be0 2024-04-25T11:38:23.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.449+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5da8068bb0 con 0x7f5da8156be0 2024-04-25T11:38:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.639+0000 7f5d8ffff640 1 -- 172.21.15.40:0/94900112 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f5d9c04b030 con 0x7f5da8156be0 2024-04-25T11:38:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-25T11:38:23.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5d9406c890 msgr2=0x7f5d9406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5d9406c890 0x7f5d9406ed50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d98007c80 tx=0x7f5d98009040 comp rx=0 tx=0).stop 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da8156be0 msgr2=0x7f5da80ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da8156be0 0x7f5da80ce600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009a90 tx=0x7f5d9c004730 comp rx=0 tx=0).stop 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 shutdown_connections 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5d9406c890 0x7f5d9406ed50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5da8155710 0x7f5da8157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da8156be0 0x7f5da80ce600 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 --2- 172.21.15.40:0/94900112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90004800 0x7f5da81f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 >> 172.21.15.40:0/94900112 conn(0x7f5da806c9a0 msgr2=0x7f5da806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 shutdown_connections 2024-04-25T11:38:23.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.641+0000 7f5dadca9640 1 -- 172.21.15.40:0/94900112 wait complete. 2024-04-25T11:38:23.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:38:23.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.764+0000 7fcb768dc640 1 Processor -- start 2024-04-25T11:38:23.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.764+0000 7fcb768dc640 1 -- start start 2024-04-25T11:38:23.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.765+0000 7fcb768dc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcb700cc820 0x7fcb701667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.765+0000 7fcb768dc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb700cbe40 con 0x7fcb70155070 2024-04-25T11:38:23.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.765+0000 7fcb768dc640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb700cb880 con 0x7fcb7015b8e0 2024-04-25T11:38:23.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.765+0000 7fcb768dc640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fcb7015be20 con 0x7fcb700cc820 2024-04-25T11:38:23.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.765+0000 7fcb6ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcb70155070 0x7fcb7015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43848/0 (socket says 172.21.15.40:43848) 2024-04-25T11:38:23.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.765+0000 7fcb6ffff640 1 -- 172.21.15.40:0/2942234188 learned_addr learned my addr 172.21.15.40:0/2942234188 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:23.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.765+0000 7fcb74e79640 1 --2- 172.21.15.40:0/2942234188 >> v2:172.21.15.62:3300/0 conn(0x7fcb700cc820 0x7fcb701667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.766+0000 7fcb6effd640 1 -- 172.21.15.40:0/2942234188 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419900227 0 0) 0x7fcb700cbe40 con 0x7fcb70155070 2024-04-25T11:38:23.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.766+0000 7fcb6effd640 1 -- 172.21.15.40:0/2942234188 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb58003520 con 0x7fcb70155070 2024-04-25T11:38:23.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.766+0000 7fcb74e79640 1 -- 172.21.15.40:0/2942234188 >> v1:172.21.15.62:6789/0 conn(0x7fcb7015b8e0 legacy=0x7fcb700d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.766+0000 7fcb74e79640 1 -- 172.21.15.40:0/2942234188 >> v1:172.21.15.40:6789/0 conn(0x7fcb70155070 legacy=0x7fcb7015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.766+0000 7fcb74e79640 1 -- 172.21.15.40:0/2942234188 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb700cbb60 con 0x7fcb700cc820 2024-04-25T11:38:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb74e79640 1 --2- 172.21.15.40:0/2942234188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb700cc820 0x7fcb701667d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009d90 tx=0x7fcb6802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7d4d7256da22147 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb6effd640 1 -- 172.21.15.40:0/2942234188 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68007740 con 0x7fcb700cc820 2024-04-25T11:38:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb6effd640 1 -- 172.21.15.40:0/2942234188 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb680078e0 con 0x7fcb700cc820 2024-04-25T11:38:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb6effd640 1 -- 172.21.15.40:0/2942234188 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68007c90 con 0x7fcb700cc820 2024-04-25T11:38:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb768dc640 1 -- 172.21.15.40:0/2942234188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb700cc820 msgr2=0x7fcb701667d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb768dc640 1 --2- 172.21.15.40:0/2942234188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb700cc820 0x7fcb701667d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009d90 tx=0x7fcb6802a660 comp rx=0 tx=0).stop 2024-04-25T11:38:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb768dc640 1 -- 172.21.15.40:0/2942234188 shutdown_connections 2024-04-25T11:38:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb768dc640 1 --2- 172.21.15.40:0/2942234188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb700cc820 0x7fcb701667d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb768dc640 1 -- 172.21.15.40:0/2942234188 >> 172.21.15.40:0/2942234188 conn(0x7fcb7006c680 msgr2=0x7fcb7006e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:23.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb768dc640 1 -- 172.21.15.40:0/2942234188 shutdown_connections 2024-04-25T11:38:23.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.767+0000 7fcb768dc640 1 -- 172.21.15.40:0/2942234188 wait complete. 2024-04-25T11:38:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.768+0000 7fcb768dc640 1 Processor -- start 2024-04-25T11:38:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb768dc640 1 -- start start 2024-04-25T11:38:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb768dc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb700cc820 0x7fcb70153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb768dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb70155070 0x7fcb70153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb768dc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb7015b8e0 0x7fcb701f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb768dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb700cb4e0 con 0x7fcb70155070 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb768dc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7015e840 con 0x7fcb700cc820 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb768dc640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcb70155450 con 0x7fcb7015b8e0 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb6f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb70155070 0x7fcb70153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb6f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb70155070 0x7fcb70153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57294/0 (socket says 172.21.15.40:57294) 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.769+0000 7fcb6f7fe640 1 -- 172.21.15.40:0/136043166 learned_addr learned my addr 172.21.15.40:0/136043166 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6ffff640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb700cc820 0x7fcb70153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb74e79640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb7015b8e0 0x7fcb701f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6f7fe640 1 -- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb7015b8e0 msgr2=0x7fcb701f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6f7fe640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb7015b8e0 0x7fcb701f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6f7fe640 1 -- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb700cc820 msgr2=0x7fcb70153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6f7fe640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb700cc820 0x7fcb70153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6f7fe640 1 -- 172.21.15.40:0/136043166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb70152c90 con 0x7fcb70155070 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6f7fe640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb70155070 0x7fcb70153c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400c440 tx=0x7fcb64001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6cff9640 1 -- 172.21.15.40:0/136043166 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb640052d0 con 0x7fcb70155070 2024-04-25T11:38:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6cff9640 1 -- 172.21.15.40:0/136043166 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb64005470 con 0x7fcb70155070 2024-04-25T11:38:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb700c5610 con 0x7fcb70155070 2024-04-25T11:38:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb6cff9640 1 -- 172.21.15.40:0/136043166 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64007260 con 0x7fcb70155070 2024-04-25T11:38:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.770+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb70152f00 con 0x7fcb70155070 2024-04-25T11:38:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.771+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb70073b30 con 0x7fcb70155070 2024-04-25T11:38:23.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.772+0000 7fcb6cff9640 1 -- 172.21.15.40:0/136043166 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb64007400 con 0x7fcb70155070 2024-04-25T11:38:23.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.772+0000 7fcb6cff9640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb5c06c8f0 0x7fcb5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:23.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.772+0000 7fcb6cff9640 1 -- 172.21.15.40:0/136043166 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fcb6408af50 con 0x7fcb70155070 2024-04-25T11:38:23.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.772+0000 7fcb6ffff640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb5c06c8f0 0x7fcb5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:23.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.773+0000 7fcb6ffff640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb5c06c8f0 0x7fcb5c06edb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb600011e0 tx=0x7fcb60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:23.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.777+0000 7fcb6cff9640 1 -- 172.21.15.40:0/136043166 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb70073b30 con 0x7fcb70155070 2024-04-25T11:38:23.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:23.963+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcb70068bb0 con 0x7fcb70155070 2024-04-25T11:38:24.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.641+0000 7fcb6cff9640 1 -- 172.21.15.40:0/136043166 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fcb640594d0 con 0x7fcb70155070 2024-04-25T11:38:24.630 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-25T11:38:24.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.645+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb5c06c8f0 msgr2=0x7fcb5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:24.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.645+0000 7fcb768dc640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb5c06c8f0 0x7fcb5c06edb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb600011e0 tx=0x7fcb60009040 comp rx=0 tx=0).stop 2024-04-25T11:38:24.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.645+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb70155070 msgr2=0x7fcb70153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:24.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.645+0000 7fcb768dc640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb70155070 0x7fcb70153c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400c440 tx=0x7fcb64001e70 comp rx=0 tx=0).stop 2024-04-25T11:38:24.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.646+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 shutdown_connections 2024-04-25T11:38:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.646+0000 7fcb768dc640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb5c06c8f0 0x7fcb5c06edb0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb600011e0 tx=0x7fcb60009040 comp rx=0 tx=0).stop 2024-04-25T11:38:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.646+0000 7fcb768dc640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb7015b8e0 0x7fcb701f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.646+0000 7fcb768dc640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb70155070 0x7fcb70153c40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.646+0000 7fcb768dc640 1 --2- 172.21.15.40:0/136043166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb700cc820 0x7fcb70153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.646+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 >> 172.21.15.40:0/136043166 conn(0x7fcb7006c680 msgr2=0x7fcb700c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.646+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 shutdown_connections 2024-04-25T11:38:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.646+0000 7fcb768dc640 1 -- 172.21.15.40:0/136043166 wait complete. 2024-04-25T11:38:24.662 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:38:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.782+0000 7fbebb259640 1 Processor -- start 2024-04-25T11:38:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.782+0000 7fbebb259640 1 -- start start 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbebb259640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb415b8e0 0x7fbeb40d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbebb259640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbeb40cc820 0x7fbeb40d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbebb259640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb40cbe40 con 0x7fbeb4155070 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbebb259640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fbeb415be20 con 0x7fbeb40cc820 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbebb259640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb40cb880 con 0x7fbeb415b8e0 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbeb8ff5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbeb4155070 0x7fbeb415ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43864/0 (socket says 172.21.15.40:43864) 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbeb8ff5640 1 -- 172.21.15.40:0/3357739366 learned_addr learned my addr 172.21.15.40:0/3357739366 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbeb97f6640 1 --2- 172.21.15.40:0/3357739366 >> v2:172.21.15.62:3300/0 conn(0x7fbeb40cc820 0x7fbeb40d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbeb3fff640 1 --2- 172.21.15.40:0/3357739366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb415b8e0 0x7fbeb40d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbeb37fe640 1 -- 172.21.15.40:0/3357739366 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 623809566 0 0) 0x7fbeb40cbe40 con 0x7fbeb4155070 2024-04-25T11:38:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.783+0000 7fbeb37fe640 1 -- 172.21.15.40:0/3357739366 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe9c003520 con 0x7fbeb4155070 2024-04-25T11:38:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.784+0000 7fbeb37fe640 1 -- 172.21.15.40:0/3357739366 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 745953226 0 0) 0x7fbe9c003520 con 0x7fbeb4155070 2024-04-25T11:38:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.784+0000 7fbeb3fff640 1 -- 172.21.15.40:0/3357739366 >> v2:172.21.15.62:3300/0 conn(0x7fbeb40cc820 msgr2=0x7fbeb40d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.784+0000 7fbeb3fff640 1 --2- 172.21.15.40:0/3357739366 >> v2:172.21.15.62:3300/0 conn(0x7fbeb40cc820 0x7fbeb40d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.784+0000 7fbeb3fff640 1 -- 172.21.15.40:0/3357739366 >> v1:172.21.15.40:6789/0 conn(0x7fbeb4155070 legacy=0x7fbeb415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.784+0000 7fbeb3fff640 1 -- 172.21.15.40:0/3357739366 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb40cbb60 con 0x7fbeb415b8e0 2024-04-25T11:38:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.784+0000 7fbeb97f6640 1 --2- 172.21.15.40:0/3357739366 >> v2:172.21.15.62:3300/0 conn(0x7fbeb40cc820 0x7fbeb40d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.784+0000 7fbeb3fff640 1 --2- 172.21.15.40:0/3357739366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb415b8e0 0x7fbeb40d04f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbea80097a0 tx=0x7fbea802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aeb55b614e43c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbeb37fe640 1 -- 172.21.15.40:0/3357739366 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8037070 con 0x7fbeb415b8e0 2024-04-25T11:38:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbeb37fe640 1 -- 172.21.15.40:0/3357739366 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea802ab10 con 0x7fbeb415b8e0 2024-04-25T11:38:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbeb37fe640 1 -- 172.21.15.40:0/3357739366 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4001670 con 0x7fbeb415b8e0 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbebb259640 1 -- 172.21.15.40:0/3357739366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb415b8e0 msgr2=0x7fbeb40d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbebb259640 1 --2- 172.21.15.40:0/3357739366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb415b8e0 0x7fbeb40d04f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbea80097a0 tx=0x7fbea802a080 comp rx=0 tx=0).stop 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbebb259640 1 -- 172.21.15.40:0/3357739366 shutdown_connections 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbebb259640 1 --2- 172.21.15.40:0/3357739366 >> v2:172.21.15.62:3300/0 conn(0x7fbeb40cc820 0x7fbeb40d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbebb259640 1 --2- 172.21.15.40:0/3357739366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb415b8e0 0x7fbeb40d04f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbebb259640 1 -- 172.21.15.40:0/3357739366 >> 172.21.15.40:0/3357739366 conn(0x7fbeb406c680 msgr2=0x7fbeb406e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbebb259640 1 -- 172.21.15.40:0/3357739366 shutdown_connections 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.785+0000 7fbebb259640 1 -- 172.21.15.40:0/3357739366 wait complete. 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.786+0000 7fbebb259640 1 Processor -- start 2024-04-25T11:38:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.786+0000 7fbebb259640 1 -- start start 2024-04-25T11:38:24.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbebb259640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb40cc820 0x7fbeb414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:24.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbebb259640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb4155070 0x7fbeb40c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:24.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbebb259640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeb415b8e0 0x7fbeb41f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:24.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbebb259640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb40cb4e0 con 0x7fbeb415b8e0 2024-04-25T11:38:24.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbebb259640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb415e840 con 0x7fbeb40cc820 2024-04-25T11:38:24.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbebb259640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb4155450 con 0x7fbeb4155070 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbeb97f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeb415b8e0 0x7fbeb41f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbeb97f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeb415b8e0 0x7fbeb41f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57306/0 (socket says 172.21.15.40:57306) 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbeb97f6640 1 -- 172.21.15.40:0/2699830557 learned_addr learned my addr 172.21.15.40:0/2699830557 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbeb3fff640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb4155070 0x7fbeb40c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.787+0000 7fbeb8ff5640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb40cc820 0x7fbeb414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbeb97f6640 1 -- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb4155070 msgr2=0x7fbeb40c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbeb97f6640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb4155070 0x7fbeb40c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbeb97f6640 1 -- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb40cc820 msgr2=0x7fbeb414fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbeb97f6640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb40cc820 0x7fbeb414fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbeb97f6640 1 -- 172.21.15.40:0/2699830557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb41f4840 con 0x7fbeb415b8e0 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbeb97f6640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeb415b8e0 0x7fbeb41f3c30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00efc0 tx=0x7fbeac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbeb17fa640 1 -- 172.21.15.40:0/2699830557 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac018030 con 0x7fbeb415b8e0 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbeb17fa640 1 -- 172.21.15.40:0/2699830557 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeac00cdf0 con 0x7fbeb415b8e0 2024-04-25T11:38:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb41f4ae0 con 0x7fbeb415b8e0 2024-04-25T11:38:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbeb17fa640 1 -- 172.21.15.40:0/2699830557 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac003bb0 con 0x7fbeb415b8e0 2024-04-25T11:38:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.788+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb41e4590 con 0x7fbeb415b8e0 2024-04-25T11:38:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.789+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb4068bb0 con 0x7fbeb415b8e0 2024-04-25T11:38:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.790+0000 7fbeb17fa640 1 -- 172.21.15.40:0/2699830557 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbeac01b070 con 0x7fbeb415b8e0 2024-04-25T11:38:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.790+0000 7fbeb17fa640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbea006c8f0 0x7fbea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:24.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.790+0000 7fbeb17fa640 1 -- 172.21.15.40:0/2699830557 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fbeac08c330 con 0x7fbeb415b8e0 2024-04-25T11:38:24.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.790+0000 7fbeb8ff5640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbea006c8f0 0x7fbea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:24.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.791+0000 7fbeb8ff5640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbea006c8f0 0x7fbea006edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbea4005880 tx=0x7fbea40057f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:24.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:24.795+0000 7fbeb17fa640 1 -- 172.21.15.40:0/2699830557 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbeac010040 con 0x7fbeb415b8e0 2024-04-25T11:38:25.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.021+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbeb40cc040 con 0x7fbeb415b8e0 2024-04-25T11:38:25.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.663+0000 7fbeb17fa640 1 -- 172.21.15.40:0/2699830557 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fbeac05a200 con 0x7fbeb415b8e0 2024-04-25T11:38:25.651 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T11:38:25.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.669+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbea006c8f0 msgr2=0x7fbea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.669+0000 7fbebb259640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbea006c8f0 0x7fbea006edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbea4005880 tx=0x7fbea40057f0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.669+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeb415b8e0 msgr2=0x7fbeb41f3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.669+0000 7fbebb259640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeb415b8e0 0x7fbeb41f3c30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00efc0 tx=0x7fbeac00c490 comp rx=0 tx=0).stop 2024-04-25T11:38:25.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.670+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 shutdown_connections 2024-04-25T11:38:25.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.670+0000 7fbebb259640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbea006c8f0 0x7fbea006edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.670+0000 7fbebb259640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbeb415b8e0 0x7fbeb41f3c30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.670+0000 7fbebb259640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbeb4155070 0x7fbeb40c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.670+0000 7fbebb259640 1 --2- 172.21.15.40:0/2699830557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb40cc820 0x7fbeb414fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.670+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 >> 172.21.15.40:0/2699830557 conn(0x7fbeb406c680 msgr2=0x7fbeb40ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.671+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 shutdown_connections 2024-04-25T11:38:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.671+0000 7fbebb259640 1 -- 172.21.15.40:0/2699830557 wait complete. 2024-04-25T11:38:25.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.795+0000 7f953d67c640 1 Processor -- start 2024-04-25T11:38:25.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.795+0000 7f953d67c640 1 -- start start 2024-04-25T11:38:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.796+0000 7f953d67c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9538155140 0x7f95380d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.796+0000 7f953d67c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95380c7ab0 0x7f95380cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.796+0000 7f953d67c640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95380d0c00 con 0x7f95380cb750 2024-04-25T11:38:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.796+0000 7f953d67c640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f953815e200 con 0x7f9538155140 2024-04-25T11:38:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.796+0000 7f953d67c640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f953815e4e0 con 0x7f95380c7ab0 2024-04-25T11:38:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.796+0000 7f95367fc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95380c7ab0 0x7f95380cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.796+0000 7f9536ffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9538155140 0x7f95380d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.796+0000 7f95377fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f95380cb750 0x7f95380cccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:54176/0 (socket says 172.21.15.40:54176) 2024-04-25T11:38:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.796+0000 7f95377fe640 1 -- 172.21.15.40:0/2073749606 learned_addr learned my addr 172.21.15.40:0/2073749606 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.797+0000 7f95367fc640 1 -- 172.21.15.40:0/2073749606 >> v2:172.21.15.62:3300/0 conn(0x7f9538155140 msgr2=0x7f95380d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.797+0000 7f95367fc640 1 --2- 172.21.15.40:0/2073749606 >> v2:172.21.15.62:3300/0 conn(0x7f9538155140 0x7f95380d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.797+0000 7f95367fc640 1 -- 172.21.15.40:0/2073749606 >> v1:172.21.15.62:6789/0 conn(0x7f95380cb750 legacy=0x7f95380cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.797+0000 7f95367fc640 1 -- 172.21.15.40:0/2073749606 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953815e7c0 con 0x7f95380c7ab0 2024-04-25T11:38:25.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.797+0000 7f9536ffd640 1 --2- 172.21.15.40:0/2073749606 >> v2:172.21.15.62:3300/0 conn(0x7f9538155140 0x7f95380d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f95367fc640 1 --2- 172.21.15.40:0/2073749606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95380c7ab0 0x7f95380cb210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f952c0097a0 tx=0x7f952c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e85f439aeb259c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f9535ffb640 1 -- 172.21.15.40:0/2073749606 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c037070 con 0x7f95380c7ab0 2024-04-25T11:38:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f9535ffb640 1 -- 172.21.15.40:0/2073749606 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c032440 con 0x7f95380c7ab0 2024-04-25T11:38:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f9535ffb640 1 -- 172.21.15.40:0/2073749606 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c032810 con 0x7f95380c7ab0 2024-04-25T11:38:25.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f953d67c640 1 -- 172.21.15.40:0/2073749606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95380c7ab0 msgr2=0x7f95380cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f953d67c640 1 --2- 172.21.15.40:0/2073749606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95380c7ab0 0x7f95380cb210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f952c0097a0 tx=0x7f952c02a180 comp rx=0 tx=0).stop 2024-04-25T11:38:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f953d67c640 1 -- 172.21.15.40:0/2073749606 shutdown_connections 2024-04-25T11:38:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f953d67c640 1 --2- 172.21.15.40:0/2073749606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95380c7ab0 0x7f95380cb210 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f953d67c640 1 --2- 172.21.15.40:0/2073749606 >> v2:172.21.15.62:3300/0 conn(0x7f9538155140 0x7f95380d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f953d67c640 1 -- 172.21.15.40:0/2073749606 >> 172.21.15.40:0/2073749606 conn(0x7f953806e320 msgr2=0x7f953806c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f953d67c640 1 -- 172.21.15.40:0/2073749606 shutdown_connections 2024-04-25T11:38:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.798+0000 7f953d67c640 1 -- 172.21.15.40:0/2073749606 wait complete. 2024-04-25T11:38:25.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.800+0000 7f953d67c640 1 Processor -- start 2024-04-25T11:38:25.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.800+0000 7f953d67c640 1 -- start start 2024-04-25T11:38:25.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.800+0000 7f953d67c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95380c7ab0 0x7f95380c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:25.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.800+0000 7f953d67c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95380cb750 0x7f95381f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:25.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.800+0000 7f953d67c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9538155140 0x7f95381f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:25.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.800+0000 7f953d67c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f953815f060 con 0x7f95380cb750 2024-04-25T11:38:25.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.800+0000 7f953d67c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f953815ed80 con 0x7f95380c7ab0 2024-04-25T11:38:25.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.800+0000 7f953d67c640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f95380d08e0 con 0x7f9538155140 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95367fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95380cb750 0x7f95381f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95367fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95380cb750 0x7f95381f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57308/0 (socket says 172.21.15.40:57308) 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95367fc640 1 -- 172.21.15.40:0/3612588412 learned_addr learned my addr 172.21.15.40:0/3612588412 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f9536ffd640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95380c7ab0 0x7f95380c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95377fe640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9538155140 0x7f95381f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95367fc640 1 -- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9538155140 msgr2=0x7f95381f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95367fc640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9538155140 0x7f95381f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95367fc640 1 -- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95380c7ab0 msgr2=0x7f95380c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95367fc640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95380c7ab0 0x7f95380c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95367fc640 1 -- 172.21.15.40:0/3612588412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95380c3f10 con 0x7f95380cb750 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f95367fc640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95380cb750 0x7f95381f4eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f952c035040 tx=0x7f952c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f951f7fe640 1 -- 172.21.15.40:0/3612588412 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c041020 con 0x7f95380cb750 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f951f7fe640 1 -- 172.21.15.40:0/3612588412 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c030700 con 0x7f95380cb750 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f951f7fe640 1 -- 172.21.15.40:0/3612588412 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c037040 con 0x7f95380cb750 2024-04-25T11:38:25.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95380c41d0 con 0x7f95380cb750 2024-04-25T11:38:25.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.801+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95380c4c30 con 0x7f95380cb750 2024-04-25T11:38:25.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.802+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95380c9d60 con 0x7f95380cb750 2024-04-25T11:38:25.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.803+0000 7f951f7fe640 1 -- 172.21.15.40:0/3612588412 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f952c044020 con 0x7f95380cb750 2024-04-25T11:38:25.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.803+0000 7f951f7fe640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f952406c920 0x7f952406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:25.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.803+0000 7f951f7fe640 1 -- 172.21.15.40:0/3612588412 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f952c0ac130 con 0x7f95380cb750 2024-04-25T11:38:25.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.803+0000 7f9536ffd640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f952406c920 0x7f952406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:25.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.804+0000 7f9536ffd640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f952406c920 0x7f952406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9528009560 tx=0x7f9528009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:25.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.808+0000 7f951f7fe640 1 -- 172.21.15.40:0/3612588412 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95380c9d60 con 0x7f95380cb750 2024-04-25T11:38:25.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.994+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9538152220 con 0x7f95380cb750 2024-04-25T11:38:25.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.995+0000 7f951f7fe640 1 -- 172.21.15.40:0/3612588412 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7f952c04b030 con 0x7f95380cb750 2024-04-25T11:38:25.984 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:25.984 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:25.023188+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:25.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f952406c920 msgr2=0x7f952406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f952406c920 0x7f952406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9528009560 tx=0x7f9528009190 comp rx=0 tx=0).stop 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95380cb750 msgr2=0x7f95381f4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95380cb750 0x7f95381f4eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f952c035040 tx=0x7f952c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 shutdown_connections 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f952406c920 0x7f952406ede0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9538155140 0x7f95381f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95380cb750 0x7f95381f4eb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 --2- 172.21.15.40:0/3612588412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95380c7ab0 0x7f95380c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 >> 172.21.15.40:0/3612588412 conn(0x7f953806e320 msgr2=0x7f95381530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 shutdown_connections 2024-04-25T11:38:25.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:25.997+0000 7f953d67c640 1 -- 172.21.15.40:0/3612588412 wait complete. 2024-04-25T11:38:25.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:26.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.109+0000 7f0d5bd6f640 1 Processor -- start 2024-04-25T11:38:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.109+0000 7f0d5bd6f640 1 -- start start 2024-04-25T11:38:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.109+0000 7f0d5bd6f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d54154f50 0x7f0d540c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d5bd6f640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d540b6210 0x7f0d540cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d5bd6f640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d5415f060 con 0x7f0d540cb7e0 2024-04-25T11:38:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d5bd6f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5415eb20 con 0x7f0d54154f50 2024-04-25T11:38:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d5bd6f640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0d540d1040 con 0x7f0d540b6210 2024-04-25T11:38:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d59b0b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d54154f50 0x7f0d540c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d59b0b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d54154f50 0x7f0d540c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57320/0 (socket says 172.21.15.40:57320) 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d59b0b640 1 -- 172.21.15.40:0/812575137 learned_addr learned my addr 172.21.15.40:0/812575137 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d5930a640 1 --2- 172.21.15.40:0/812575137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d540b6210 0x7f0d540cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d59b0b640 1 -- 172.21.15.40:0/812575137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d540b6210 msgr2=0x7f0d540cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d59b0b640 1 --2- 172.21.15.40:0/812575137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d540b6210 0x7f0d540cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d59b0b640 1 -- 172.21.15.40:0/812575137 >> v1:172.21.15.62:6789/0 conn(0x7f0d540cb7e0 legacy=0x7f0d540c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.110+0000 7f0d59b0b640 1 -- 172.21.15.40:0/812575137 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d540cc150 con 0x7f0d54154f50 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d59b0b640 1 --2- 172.21.15.40:0/812575137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d54154f50 0x7f0d540c7a70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009870 tx=0x7f0d5002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa3437e50bf0603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d58b09640 1 -- 172.21.15.40:0/812575137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50037070 con 0x7f0d54154f50 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d58b09640 1 -- 172.21.15.40:0/812575137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d50032440 con 0x7f0d54154f50 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d58b09640 1 -- 172.21.15.40:0/812575137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50032810 con 0x7f0d54154f50 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/812575137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d54154f50 msgr2=0x7f0d540c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d5bd6f640 1 --2- 172.21.15.40:0/812575137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d54154f50 0x7f0d540c7a70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009870 tx=0x7f0d5002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/812575137 shutdown_connections 2024-04-25T11:38:26.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d5bd6f640 1 --2- 172.21.15.40:0/812575137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d540b6210 0x7f0d540cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d5bd6f640 1 --2- 172.21.15.40:0/812575137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d54154f50 0x7f0d540c7a70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/812575137 >> 172.21.15.40:0/812575137 conn(0x7f0d540d0440 msgr2=0x7f0d540d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:26.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.111+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/812575137 shutdown_connections 2024-04-25T11:38:26.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.112+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/812575137 wait complete. 2024-04-25T11:38:26.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d5bd6f640 1 Processor -- start 2024-04-25T11:38:26.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d5bd6f640 1 -- start start 2024-04-25T11:38:26.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d5bd6f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d540cb7e0 0x7f0d54157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d5bd6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d54154f50 0x7f0d540cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d5bd6f640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d5415dae0 0x7f0d54155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d5bd6f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d540cc3f0 con 0x7f0d540cb7e0 2024-04-25T11:38:26.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d5bd6f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5415ed80 con 0x7f0d54154f50 2024-04-25T11:38:26.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d5bd6f640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0d540c8170 con 0x7f0d5415dae0 2024-04-25T11:38:26.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d59b0b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d540cb7e0 0x7f0d54157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d59b0b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d540cb7e0 0x7f0d54157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57326/0 (socket says 172.21.15.40:57326) 2024-04-25T11:38:26.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.113+0000 7f0d59b0b640 1 -- 172.21.15.40:0/1384064050 learned_addr learned my addr 172.21.15.40:0/1384064050 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:26.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d59b0b640 1 -- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d5415dae0 msgr2=0x7f0d54155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d59b0b640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d5415dae0 0x7f0d54155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d59b0b640 1 -- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d54154f50 msgr2=0x7f0d540cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d59b0b640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d54154f50 0x7f0d540cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d59b0b640 1 -- 172.21.15.40:0/1384064050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d540c9040 con 0x7f0d540cb7e0 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d59b0b640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d540cb7e0 0x7f0d54157600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0d50000c00 tx=0x7f0d500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d4a7fc640 1 -- 172.21.15.40:0/1384064050 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50041020 con 0x7f0d540cb7e0 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d4a7fc640 1 -- 172.21.15.40:0/1384064050 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d500306c0 con 0x7f0d540cb7e0 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d540c9320 con 0x7f0d540cb7e0 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d4a7fc640 1 -- 172.21.15.40:0/1384064050 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50037040 con 0x7f0d540cb7e0 2024-04-25T11:38:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.114+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d541e7500 con 0x7f0d540cb7e0 2024-04-25T11:38:26.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.115+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d54068bb0 con 0x7f0d540cb7e0 2024-04-25T11:38:26.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.115+0000 7f0d4a7fc640 1 -- 172.21.15.40:0/1384064050 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d50044020 con 0x7f0d540cb7e0 2024-04-25T11:38:26.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.116+0000 7f0d4a7fc640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d3806c8f0 0x7f0d3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.116+0000 7f0d4a7fc640 1 -- 172.21.15.40:0/1384064050 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f0d500ac3f0 con 0x7f0d540cb7e0 2024-04-25T11:38:26.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.116+0000 7f0d5930a640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d3806c8f0 0x7f0d3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.116+0000 7f0d5930a640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d3806c8f0 0x7f0d3806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0d44006fd0 tx=0x7f0d44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.120+0000 7f0d4a7fc640 1 -- 172.21.15.40:0/1384064050 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d5004b030 con 0x7f0d540cb7e0 2024-04-25T11:38:26.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.306+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d540b5dc0 con 0x7f0d540cb7e0 2024-04-25T11:38:26.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.307+0000 7f0d4a7fc640 1 -- 172.21.15.40:0/1384064050 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7f0d5007a820 con 0x7f0d540cb7e0 2024-04-25T11:38:26.295 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:26.296 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:25.023188+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:26.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d3806c8f0 msgr2=0x7f0d3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d3806c8f0 0x7f0d3806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0d44006fd0 tx=0x7f0d44008040 comp rx=0 tx=0).stop 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d540cb7e0 msgr2=0x7f0d54157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d540cb7e0 0x7f0d54157600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0d50000c00 tx=0x7f0d500046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 shutdown_connections 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d3806c8f0 0x7f0d3806edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d5415dae0 0x7f0d54155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d54154f50 0x7f0d540cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 --2- 172.21.15.40:0/1384064050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d540cb7e0 0x7f0d54157600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 >> 172.21.15.40:0/1384064050 conn(0x7f0d540d0440 msgr2=0x7f0d540c7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 shutdown_connections 2024-04-25T11:38:26.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.309+0000 7f0d5bd6f640 1 -- 172.21.15.40:0/1384064050 wait complete. 2024-04-25T11:38:26.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:38:26.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.424+0000 7f03d5994640 1 Processor -- start 2024-04-25T11:38:26.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.424+0000 7f03d5994640 1 -- start start 2024-04-25T11:38:26.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.424+0000 7f03d5994640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 0x7f03d015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.424+0000 7f03d5994640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03d015b8e0 0x7f03d00d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.424+0000 7f03d5994640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f03d00cc820 0x7f03d00d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.424+0000 7f03d5994640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f03d015be20 con 0x7f03d015b8e0 2024-04-25T11:38:26.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.424+0000 7f03d5994640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f03d00cbe40 con 0x7f03d00cc820 2024-04-25T11:38:26.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.424+0000 7f03d5994640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f03d00cb880 con 0x7f03d0155070 2024-04-25T11:38:26.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.424+0000 7f03ce7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03d015b8e0 0x7f03d00d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03ce7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03d015b8e0 0x7f03d00d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57336/0 (socket says 172.21.15.40:57336) 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03ce7fc640 1 -- 172.21.15.40:0/689066955 learned_addr learned my addr 172.21.15.40:0/689066955 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03ceffd640 1 --2- 172.21.15.40:0/689066955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 0x7f03d015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03cf7fe640 1 --2- 172.21.15.40:0/689066955 >> v2:172.21.15.62:3300/0 conn(0x7f03d00cc820 0x7f03d00d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03ce7fc640 1 -- 172.21.15.40:0/689066955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 msgr2=0x7f03d015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03ce7fc640 1 --2- 172.21.15.40:0/689066955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 0x7f03d015ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03ce7fc640 1 -- 172.21.15.40:0/689066955 >> v2:172.21.15.62:3300/0 conn(0x7f03d00cc820 msgr2=0x7f03d00d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03ce7fc640 1 --2- 172.21.15.40:0/689066955 >> v2:172.21.15.62:3300/0 conn(0x7f03d00cc820 0x7f03d00d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03ce7fc640 1 -- 172.21.15.40:0/689066955 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d00cbb60 con 0x7f03d015b8e0 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.425+0000 7f03ce7fc640 1 --2- 172.21.15.40:0/689066955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d015b8e0 0x7f03d00d04f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b5da0 tx=0x7f03c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b2621755c4dae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03cdffb640 1 -- 172.21.15.40:0/689066955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0037070 con 0x7f03d015b8e0 2024-04-25T11:38:26.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03cdffb640 1 -- 172.21.15.40:0/689066955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c0032440 con 0x7f03d015b8e0 2024-04-25T11:38:26.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03cdffb640 1 -- 172.21.15.40:0/689066955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0032810 con 0x7f03d015b8e0 2024-04-25T11:38:26.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03d5994640 1 -- 172.21.15.40:0/689066955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d015b8e0 msgr2=0x7f03d00d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03d5994640 1 --2- 172.21.15.40:0/689066955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d015b8e0 0x7f03d00d04f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b5da0 tx=0x7f03c002a140 comp rx=0 tx=0).stop 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03d5994640 1 -- 172.21.15.40:0/689066955 shutdown_connections 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03d5994640 1 --2- 172.21.15.40:0/689066955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d015b8e0 0x7f03d00d04f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03d5994640 1 --2- 172.21.15.40:0/689066955 >> v2:172.21.15.62:3300/0 conn(0x7f03d00cc820 0x7f03d00d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03d5994640 1 --2- 172.21.15.40:0/689066955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 0x7f03d015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03d5994640 1 -- 172.21.15.40:0/689066955 >> 172.21.15.40:0/689066955 conn(0x7f03d006c680 msgr2=0x7f03d006e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03d5994640 1 -- 172.21.15.40:0/689066955 shutdown_connections 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.426+0000 7f03d5994640 1 -- 172.21.15.40:0/689066955 wait complete. 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.427+0000 7f03d5994640 1 Processor -- start 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03d5994640 1 -- start start 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03d5994640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d00cc820 0x7f03d01e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03d5994640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 0x7f03d015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03d5994640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d015b8e0 0x7f03d01f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03d5994640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03d00cb4e0 con 0x7f03d00cc820 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03d5994640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f03d015e840 con 0x7f03d015b8e0 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03d5994640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f03d015f2a0 con 0x7f03d0155070 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03ceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d00cc820 0x7f03d01e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03ceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d00cc820 0x7f03d01e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57338/0 (socket says 172.21.15.40:57338) 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03ceffd640 1 -- 172.21.15.40:0/1554469520 learned_addr learned my addr 172.21.15.40:0/1554469520 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03cf7fe640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d015b8e0 0x7f03d01f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.428+0000 7f03ce7fc640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 0x7f03d015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03ceffd640 1 -- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 msgr2=0x7f03d015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03ceffd640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 0x7f03d015a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03ceffd640 1 -- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d015b8e0 msgr2=0x7f03d01f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03ceffd640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d015b8e0 0x7f03d01f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03ceffd640 1 -- 172.21.15.40:0/1554469520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d01e8170 con 0x7f03d00cc820 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03ceffd640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d00cc820 0x7f03d01e63d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03b800c7f0 tx=0x7f03b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03c7fff640 1 -- 172.21.15.40:0/1554469520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8015070 con 0x7f03d00cc820 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03c7fff640 1 -- 172.21.15.40:0/1554469520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b8004220 con 0x7f03d00cc820 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d01f0ec0 con 0x7f03d00cc820 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03c7fff640 1 -- 172.21.15.40:0/1554469520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8003610 con 0x7f03d00cc820 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.429+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d01f1480 con 0x7f03d00cc820 2024-04-25T11:38:26.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.430+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d0068bb0 con 0x7f03d00cc820 2024-04-25T11:38:26.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.430+0000 7f03c7fff640 1 -- 172.21.15.40:0/1554469520 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03b80039c0 con 0x7f03d00cc820 2024-04-25T11:38:26.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.431+0000 7f03c7fff640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03b006c920 0x7f03b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.431+0000 7f03c7fff640 1 -- 172.21.15.40:0/1554469520 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f03b808baf0 con 0x7f03d00cc820 2024-04-25T11:38:26.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.431+0000 7f03ce7fc640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03b006c920 0x7f03b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.431+0000 7f03ce7fc640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03b006c920 0x7f03b006ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f03c0034000 tx=0x7f03c002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.435+0000 7f03c7fff640 1 -- 172.21.15.40:0/1554469520 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03b8059f10 con 0x7f03d00cc820 2024-04-25T11:38:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.661+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f03d00b6160 con 0x7f03d00cc820 2024-04-25T11:38:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.669+0000 7f03c7fff640 1 -- 172.21.15.40:0/1554469520 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7f03b8059810 con 0x7f03d00cc820 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03b006c920 msgr2=0x7f03b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03b006c920 0x7f03b006ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f03c0034000 tx=0x7f03c002e000 comp rx=0 tx=0).stop 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d00cc820 msgr2=0x7f03d01e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d00cc820 0x7f03d01e63d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03b800c7f0 tx=0x7f03b800ccb0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 shutdown_connections 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03b006c920 0x7f03b006ede0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03d015b8e0 0x7f03d01f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03d0155070 0x7f03d015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 --2- 172.21.15.40:0/1554469520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d00cc820 0x7f03d01e63d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 >> 172.21.15.40:0/1554469520 conn(0x7f03d006c680 msgr2=0x7f03d00ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 shutdown_connections 2024-04-25T11:38:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.672+0000 7f03d5994640 1 -- 172.21.15.40:0/1554469520 wait complete. 2024-04-25T11:38:26.671 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2024-04-25T11:38:26.671 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T11:38:26.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.787+0000 7f15df433640 1 Processor -- start 2024-04-25T11:38:26.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.788+0000 7f15df433640 1 -- start start 2024-04-25T11:38:26.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.788+0000 7f15df433640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f15d80cb2a0 0x7f15d81647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.788+0000 7f15df433640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d80b1080 con 0x7f15d80ca740 2024-04-25T11:38:26.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.788+0000 7f15df433640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d80cb730 con 0x7f15d80caec0 2024-04-25T11:38:26.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.788+0000 7f15df433640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f15d80c92d0 con 0x7f15d80cb2a0 2024-04-25T11:38:26.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.788+0000 7f15dc9ce640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f15d80ca740 0x7f15d806f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43870/0 (socket says 172.21.15.40:43870) 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.788+0000 7f15dc9ce640 1 -- 172.21.15.40:0/2323447209 learned_addr learned my addr 172.21.15.40:0/2323447209 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3369727330 0 0) 0x7f15d80b1080 con 0x7f15d80ca740 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15c4003520 con 0x7f15d80ca740 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15dd9d0640 1 --2- 172.21.15.40:0/2323447209 >> v2:172.21.15.62:3300/0 conn(0x7f15d80cb2a0 0x7f15d81647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3025871155 0 0) 0x7f15c4003520 con 0x7f15d80ca740 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 >> v2:172.21.15.62:3300/0 conn(0x7f15d80cb2a0 msgr2=0x7f15d81647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 --2- 172.21.15.40:0/2323447209 >> v2:172.21.15.62:3300/0 conn(0x7f15d80cb2a0 0x7f15d81647d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 >> v1:172.21.15.62:6789/0 conn(0x7f15d80caec0 legacy=0x7f15d8159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d80c9760 con 0x7f15d80ca740 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7f15c80017f0 con 0x7f15d80ca740 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 >> v1:172.21.15.40:6789/0 conn(0x7f15d80ca740 legacy=0x7f15d806f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15c40039e0 0x7f15c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15c4004800 0x7f15c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c4005210 0x7f15c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80c92d0 con 0x7f15c4004800 2024-04-25T11:38:26.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15c400c140 con 0x7f15c4005210 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f15c400c340 con 0x7f15c40039e0 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15dc9ce640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15c4004800 0x7f15c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15dd9d0640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c4005210 0x7f15c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.789+0000 7f15dd1cf640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15c40039e0 0x7f15c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15dc9ce640 1 -- 172.21.15.40:0/2323447209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15c40039e0 msgr2=0x7f15c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15dc9ce640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15c40039e0 0x7f15c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15dc9ce640 1 -- 172.21.15.40:0/2323447209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c4005210 msgr2=0x7f15c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15dc9ce640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c4005210 0x7f15c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15dc9ce640 1 -- 172.21.15.40:0/2323447209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c4003520 con 0x7f15c4004800 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15dd9d0640 1 -- 172.21.15.40:0/2323447209 reap_dead start 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15dc9ce640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15c4004800 0x7f15c4004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c160 tx=0x7f15c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e6de9e2ca9ed16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8007cd0 con 0x7f15c4004800 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c8007290 con 0x7f15c4004800 2024-04-25T11:38:26.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.790+0000 7f15cffff640 1 -- 172.21.15.40:0/2323447209 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c80057a0 con 0x7f15c4004800 2024-04-25T11:38:26.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.791+0000 7f15df433640 1 -- 172.21.15.40:0/2323447209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15c4004800 msgr2=0x7f15c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.791+0000 7f15df433640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15c4004800 0x7f15c4004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c160 tx=0x7f15c4005830 comp rx=0 tx=0).stop 2024-04-25T11:38:26.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.791+0000 7f15df433640 1 -- 172.21.15.40:0/2323447209 shutdown_connections 2024-04-25T11:38:26.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.791+0000 7f15df433640 1 --2- 172.21.15.40:0/2323447209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15c4004800 0x7f15c4004c80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.791+0000 7f15df433640 1 -- 172.21.15.40:0/2323447209 >> 172.21.15.40:0/2323447209 conn(0x7f15d80681a0 msgr2=0x7f15d80714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:26.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.791+0000 7f15df433640 1 -- 172.21.15.40:0/2323447209 shutdown_connections 2024-04-25T11:38:26.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.791+0000 7f15df433640 1 -- 172.21.15.40:0/2323447209 wait complete. 2024-04-25T11:38:26.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.792+0000 7f15df433640 1 Processor -- start 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.792+0000 7f15df433640 1 -- start start 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15df433640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c4004800 0x7f15d81497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15df433640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80b0050 0x7f15d806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15df433640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15d806d680 0x7f15d80ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15df433640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80ca0f0 con 0x7f15d80b0050 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15df433640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80c9e10 con 0x7f15c4004800 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15df433640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f15d80c3450 con 0x7f15d806d680 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dc9ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80b0050 0x7f15d806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dc9ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80b0050 0x7f15d806d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57354/0 (socket says 172.21.15.40:57354) 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dc9ce640 1 -- 172.21.15.40:0/3110383239 learned_addr learned my addr 172.21.15.40:0/3110383239 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dd1cf640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c4004800 0x7f15d81497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dd9d0640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15d806d680 0x7f15d80ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dc9ce640 1 -- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15d806d680 msgr2=0x7f15d80ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dc9ce640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15d806d680 0x7f15d80ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dc9ce640 1 -- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c4004800 msgr2=0x7f15d81497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dc9ce640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c4004800 0x7f15d81497d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.793+0000 7f15dc9ce640 1 -- 172.21.15.40:0/3110383239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d806db00 con 0x7f15d80b0050 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.794+0000 7f15dc9ce640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80b0050 0x7f15d806d140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c160 tx=0x7f15d80c3cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.794+0000 7f15cdffb640 1 -- 172.21.15.40:0/3110383239 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8040020 con 0x7f15d80b0050 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.794+0000 7f15cdffb640 1 -- 172.21.15.40:0/3110383239 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c80015b0 con 0x7f15d80b0050 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.794+0000 7f15cdffb640 1 -- 172.21.15.40:0/3110383239 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8037040 con 0x7f15d80b0050 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.794+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d81487a0 con 0x7f15d80b0050 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.794+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d8148d60 con 0x7f15d80b0050 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.795+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d8069200 con 0x7f15d80b0050 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.795+0000 7f15cdffb640 1 -- 172.21.15.40:0/3110383239 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15c8043020 con 0x7f15d80b0050 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.796+0000 7f15cdffb640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f15bc06c8f0 0x7f15bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.796+0000 7f15cdffb640 1 -- 172.21.15.40:0/3110383239 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f15c80ab4c0 con 0x7f15d80b0050 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.796+0000 7f15dd1cf640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f15bc06c8f0 0x7f15bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.796+0000 7f15dd1cf640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f15bc06c8f0 0x7f15bc06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15d4005510 tx=0x7f15d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:26.800+0000 7f15cdffb640 1 -- 172.21.15.40:0/3110383239 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15c804a070 con 0x7f15d80b0050 2024-04-25T11:38:27.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.023+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f15d80b1080 con 0x7f15d80b0050 2024-04-25T11:38:27.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:38:27.640 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:38:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.674+0000 7f15cdffb640 1 -- 172.21.15.40:0/3110383239 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7f15d806db00 con 0x7f15d80b0050 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f15bc06c8f0 msgr2=0x7f15bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f15bc06c8f0 0x7f15bc06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15d4005510 tx=0x7f15d4009040 comp rx=0 tx=0).stop 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80b0050 msgr2=0x7f15d806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80b0050 0x7f15d806d140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c160 tx=0x7f15d80c3cd0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 shutdown_connections 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f15bc06c8f0 0x7f15bc06edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15d806d680 0x7f15d80ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80b0050 0x7f15d806d140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 --2- 172.21.15.40:0/3110383239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c4004800 0x7f15d81497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 >> 172.21.15.40:0/3110383239 conn(0x7f15d80681a0 msgr2=0x7f15d8155700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.678+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 shutdown_connections 2024-04-25T11:38:27.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.679+0000 7f15df433640 1 -- 172.21.15.40:0/3110383239 wait complete. 2024-04-25T11:38:27.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:38:27.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.869+0000 7f1db4756640 1 Processor -- start 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.869+0000 7f1db4756640 1 -- start start 2024-04-25T11:38:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.870+0000 7f1db4756640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac0cd2d0 0x7f1dac157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.870+0000 7f1db4756640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1dac070400 0x7f1dac0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.870+0000 7f1db4756640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dac1582c0 con 0x7f1dac152090 2024-04-25T11:38:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.870+0000 7f1db4756640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1dac15b6c0 con 0x7f1dac070400 2024-04-25T11:38:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.870+0000 7f1db4756640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac158580 con 0x7f1dac0cd2d0 2024-04-25T11:38:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.870+0000 7f1db2cf3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1dac070400 0x7f1dac0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.870+0000 7f1db2cf3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1dac070400 0x7f1dac0c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57370/0 (socket says 172.21.15.40:57370) 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.870+0000 7f1db2cf3640 1 -- 172.21.15.40:0/3965530737 learned_addr learned my addr 172.21.15.40:0/3965530737 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.871+0000 7f1db1cf1640 1 --2- 172.21.15.40:0/3965530737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac0cd2d0 0x7f1dac157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.871+0000 7f1db2cf3640 1 -- 172.21.15.40:0/3965530737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac0cd2d0 msgr2=0x7f1dac157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.871+0000 7f1db2cf3640 1 --2- 172.21.15.40:0/3965530737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac0cd2d0 0x7f1dac157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.871+0000 7f1db2cf3640 1 -- 172.21.15.40:0/3965530737 >> v1:172.21.15.62:6789/0 conn(0x7f1dac152090 legacy=0x7f1dac15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.871+0000 7f1db2cf3640 1 -- 172.21.15.40:0/3965530737 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac158860 con 0x7f1dac070400 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.871+0000 7f1db2cf3640 1 --2- 172.21.15.40:0/3965530737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac070400 0x7f1dac0c9d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009e50 tx=0x7f1d9c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fa7ef509637dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db14f0640 1 -- 172.21.15.40:0/3965530737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c036070 con 0x7f1dac070400 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db14f0640 1 -- 172.21.15.40:0/3965530737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d9c031600 con 0x7f1dac070400 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db14f0640 1 -- 172.21.15.40:0/3965530737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0319d0 con 0x7f1dac070400 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db4756640 1 -- 172.21.15.40:0/3965530737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac070400 msgr2=0x7f1dac0c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db4756640 1 --2- 172.21.15.40:0/3965530737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac070400 0x7f1dac0c9d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009e50 tx=0x7f1d9c02a710 comp rx=0 tx=0).stop 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db4756640 1 -- 172.21.15.40:0/3965530737 shutdown_connections 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db4756640 1 --2- 172.21.15.40:0/3965530737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac070400 0x7f1dac0c9d60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db4756640 1 --2- 172.21.15.40:0/3965530737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac0cd2d0 0x7f1dac157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db4756640 1 -- 172.21.15.40:0/3965530737 >> 172.21.15.40:0/3965530737 conn(0x7f1dac0676b0 msgr2=0x7f1dac067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db4756640 1 -- 172.21.15.40:0/3965530737 shutdown_connections 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.872+0000 7f1db4756640 1 -- 172.21.15.40:0/3965530737 wait complete. 2024-04-25T11:38:27.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.873+0000 7f1db4756640 1 Processor -- start 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db4756640 1 -- start start 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db4756640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac070400 0x7f1dac0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db4756640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dac0cd2d0 0x7f1dac1f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db4756640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac152090 0x7f1dac155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db4756640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac070870 con 0x7f1dac152090 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db4756640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac070cc0 con 0x7f1dac0cd2d0 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db4756640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac0709f0 con 0x7f1dac070400 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db2cf3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac152090 0x7f1dac155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db2cf3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac152090 0x7f1dac155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57386/0 (socket says 172.21.15.40:57386) 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.874+0000 7f1db2cf3640 1 -- 172.21.15.40:0/1870700366 learned_addr learned my addr 172.21.15.40:0/1870700366 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db24f2640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac070400 0x7f1dac0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db1cf1640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dac0cd2d0 0x7f1dac1f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db2cf3640 1 -- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac070400 msgr2=0x7f1dac0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db2cf3640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac070400 0x7f1dac0c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db2cf3640 1 -- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dac0cd2d0 msgr2=0x7f1dac1f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db2cf3640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dac0cd2d0 0x7f1dac1f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db2cf3640 1 -- 172.21.15.40:0/1870700366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac1f2840 con 0x7f1dac152090 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db1cf1640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dac0cd2d0 0x7f1dac1f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db2cf3640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac152090 0x7f1dac155ca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009e50 tx=0x7f1d9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1daaffd640 1 -- 172.21.15.40:0/1870700366 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c036050 con 0x7f1dac152090 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1daaffd640 1 -- 172.21.15.40:0/1870700366 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d9c042070 con 0x7f1dac152090 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac1f2b00 con 0x7f1dac152090 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1daaffd640 1 -- 172.21.15.40:0/1870700366 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c009510 con 0x7f1dac152090 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.875+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac0c95f0 con 0x7f1dac152090 2024-04-25T11:38:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.876+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac0c9920 con 0x7f1dac152090 2024-04-25T11:38:27.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.876+0000 7f1daaffd640 1 -- 172.21.15.40:0/1870700366 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d9c007bd0 con 0x7f1dac152090 2024-04-25T11:38:27.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.877+0000 7f1daaffd640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1d9806c920 0x7f1d9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.877+0000 7f1daaffd640 1 -- 172.21.15.40:0/1870700366 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1d9c0ac9d0 con 0x7f1dac152090 2024-04-25T11:38:27.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.877+0000 7f1db24f2640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1d9806c920 0x7f1d9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.878+0000 7f1db24f2640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1d9806c920 0x7f1d9806ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1da4007fd0 tx=0x7f1da4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:27.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:27.881+0000 7f1daaffd640 1 -- 172.21.15.40:0/1870700366 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d9c07ae00 con 0x7f1dac152090 2024-04-25T11:38:28.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.109+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1dac069200 con 0x7f1dac152090 2024-04-25T11:38:28.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.676+0000 7f1daaffd640 1 -- 172.21.15.40:0/1870700366 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f1d9c07a700 con 0x7f1dac152090 2024-04-25T11:38:28.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1d9806c920 msgr2=0x7f1d9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1d9806c920 0x7f1d9806ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1da4007fd0 tx=0x7f1da4009040 comp rx=0 tx=0).stop 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac152090 msgr2=0x7f1dac155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac152090 0x7f1dac155ca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009e50 tx=0x7f1d9c004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 shutdown_connections 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1d9806c920 0x7f1d9806ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dac152090 0x7f1dac155ca0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dac0cd2d0 0x7f1dac1f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 --2- 172.21.15.40:0/1870700366 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1dac070400 0x7f1dac0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 >> 172.21.15.40:0/1870700366 conn(0x7f1dac0676b0 msgr2=0x7f1dac1574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 shutdown_connections 2024-04-25T11:38:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.679+0000 7f1db4756640 1 -- 172.21.15.40:0/1870700366 wait complete. 2024-04-25T11:38:28.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T11:38:28.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.802+0000 7f07a8108640 1 Processor -- start 2024-04-25T11:38:28.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.803+0000 7f07a8108640 1 -- start start 2024-04-25T11:38:28.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.803+0000 7f07a8108640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f07a01550f0 0x7f07a00c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.803+0000 7f07a8108640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07a015e760 0x7f07a01656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.803+0000 7f07a8108640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a0070010 con 0x7f07a015e310 2024-04-25T11:38:28.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.803+0000 7f07a8108640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f07a015bce0 con 0x7f07a01550f0 2024-04-25T11:38:28.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.803+0000 7f07a8108640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f07a006fd50 con 0x7f07a015e760 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.803+0000 7f07a56a3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f07a015e310 0x7f07a00c9990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43872/0 (socket says 172.21.15.40:43872) 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.803+0000 7f07a56a3640 1 -- 172.21.15.40:0/2571211872 learned_addr learned my addr 172.21.15.40:0/2571211872 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a5ea4640 1 --2- 172.21.15.40:0/2571211872 >> v2:172.21.15.62:3300/0 conn(0x7f07a01550f0 0x7f07a00c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a66a5640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07a015e760 0x7f07a01656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321941290 0 0) 0x7f07a0070010 con 0x7f07a015e310 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f078c003520 con 0x7f07a015e310 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2638842896 0 0) 0x7f078c003520 con 0x7f07a015e310 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07a015e760 msgr2=0x7f07a01656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07a015e760 0x7f07a01656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 >> v2:172.21.15.62:3300/0 conn(0x7f07a01550f0 msgr2=0x7f07a00c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 --2- 172.21.15.40:0/2571211872 >> v2:172.21.15.62:3300/0 conn(0x7f07a01550f0 0x7f07a00c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a01554d0 con 0x7f07a015e310 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7f0790001670 con 0x7f07a015e310 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a5ea4640 1 --2- 172.21.15.40:0/2571211872 >> v2:172.21.15.62:3300/0 conn(0x7f07a01550f0 0x7f07a00c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 >> v1:172.21.15.40:6789/0 conn(0x7f07a015e310 legacy=0x7f07a00c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0039b0 0x7f078c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07a015e760 0x7f078c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c004d40 0x7f078c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f07a015bce0 con 0x7f078c004d40 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07a006fd50 con 0x7f078c0039b0 2024-04-25T11:38:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f078c00b9c0 con 0x7f07a015e760 2024-04-25T11:38:28.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a66a5640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c004d40 0x7f078c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a56a3640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07a015e760 0x7f078c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.804+0000 7f07a5ea4640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0039b0 0x7f078c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a66a5640 1 -- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07a015e760 msgr2=0x7f078c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:28.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a66a5640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07a015e760 0x7f078c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a66a5640 1 -- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0039b0 msgr2=0x7f078c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:28.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a66a5640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0039b0 0x7f078c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a66a5640 1 -- 172.21.15.40:0/2571211872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078c003520 con 0x7f078c004d40 2024-04-25T11:38:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a56a3640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07a015e760 0x7f078c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a5ea4640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0039b0 0x7f078c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a66a5640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c004d40 0x7f078c0051c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0798009fc0 tx=0x7f078c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13a5fcc99e81998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798009c10 con 0x7f078c004d40 2024-04-25T11:38:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0798036070 con 0x7f078c004d40 2024-04-25T11:38:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.805+0000 7f07a4ea2640 1 -- 172.21.15.40:0/2571211872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798031880 con 0x7f078c004d40 2024-04-25T11:38:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.806+0000 7f07a8108640 1 -- 172.21.15.40:0/2571211872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c004d40 msgr2=0x7f078c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.806+0000 7f07a8108640 1 --2- 172.21.15.40:0/2571211872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c004d40 0x7f078c0051c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0798009fc0 tx=0x7f078c005830 comp rx=0 tx=0).stop 2024-04-25T11:38:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.806+0000 7f07a66a5640 1 -- 172.21.15.40:0/2571211872 reap_dead start 2024-04-25T11:38:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.806+0000 7f07a8108640 1 -- 172.21.15.40:0/2571211872 shutdown_connections 2024-04-25T11:38:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.806+0000 7f07a8108640 1 -- 172.21.15.40:0/2571211872 >> 172.21.15.40:0/2571211872 conn(0x7f07a006bfa0 msgr2=0x7f07a00676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.806+0000 7f07a8108640 1 -- 172.21.15.40:0/2571211872 shutdown_connections 2024-04-25T11:38:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.806+0000 7f07a8108640 1 -- 172.21.15.40:0/2571211872 wait complete. 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.807+0000 7f07a8108640 1 Processor -- start 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.807+0000 7f07a8108640 1 -- start start 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a8108640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f078c004d40 0x7f07a014e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a8108640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07a00c8e30 0x7f07a00c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a8108640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07a014aee0 0x7f07a014b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a8108640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f07a015b4e0 con 0x7f07a014aee0 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a8108640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07a015b200 con 0x7f07a00c8e30 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a8108640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f07a015b7c0 con 0x7f078c004d40 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a66a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07a014aee0 0x7f07a014b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a66a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07a014aee0 0x7f07a014b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57418/0 (socket says 172.21.15.40:57418) 2024-04-25T11:38:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a66a5640 1 -- 172.21.15.40:0/2681155563 learned_addr learned my addr 172.21.15.40:0/2681155563 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a56a3640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07a00c8e30 0x7f07a00c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.808+0000 7f07a5ea4640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f078c004d40 0x7f07a014e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07a66a5640 1 -- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f078c004d40 msgr2=0x7f07a014e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07a66a5640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f078c004d40 0x7f07a014e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07a66a5640 1 -- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07a00c8e30 msgr2=0x7f07a00c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07a66a5640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07a00c8e30 0x7f07a00c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07a66a5640 1 -- 172.21.15.40:0/2681155563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a00c97f0 con 0x7f07a014aee0 2024-04-25T11:38:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07a56a3640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07a00c8e30 0x7f07a00c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07a66a5640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07a014aee0 0x7f07a014b360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f079802aee0 tx=0x7f0798009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07967fc640 1 -- 172.21.15.40:0/2681155563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c009860 con 0x7f07a014aee0 2024-04-25T11:38:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07967fc640 1 -- 172.21.15.40:0/2681155563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0798040030 con 0x7f07a014aee0 2024-04-25T11:38:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a014f190 con 0x7f07a014aee0 2024-04-25T11:38:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07967fc640 1 -- 172.21.15.40:0/2681155563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790007d50 con 0x7f07a014aee0 2024-04-25T11:38:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.809+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a01e55f0 con 0x7f07a014aee0 2024-04-25T11:38:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.810+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a0069200 con 0x7f07a014aee0 2024-04-25T11:38:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.810+0000 7f07967fc640 1 -- 172.21.15.40:0/2681155563 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0798005d10 con 0x7f07a014aee0 2024-04-25T11:38:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.811+0000 7f07967fc640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f078406c8f0 0x7f078406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.811+0000 7f07967fc640 1 -- 172.21.15.40:0/2681155563 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07980aca20 con 0x7f07a014aee0 2024-04-25T11:38:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.811+0000 7f07a5ea4640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f078406c8f0 0x7f078406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.812+0000 7f07a5ea4640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f078406c8f0 0x7f078406edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f079c009ef0 tx=0x7f079c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:28.815+0000 7f07967fc640 1 -- 172.21.15.40:0/2681155563 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f079804a020 con 0x7f07a014aee0 2024-04-25T11:38:29.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.040+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f07a00701e0 con 0x7f07a014aee0 2024-04-25T11:38:29.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.685+0000 7f07967fc640 1 -- 172.21.15.40:0/2681155563 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7f07a00c97f0 con 0x7f07a014aee0 2024-04-25T11:38:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f078406c8f0 msgr2=0x7f078406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f078406c8f0 0x7f078406edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f079c009ef0 tx=0x7f079c009190 comp rx=0 tx=0).stop 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07a014aee0 msgr2=0x7f07a014b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07a014aee0 0x7f07a014b360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f079802aee0 tx=0x7f0798009750 comp rx=0 tx=0).stop 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 shutdown_connections 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f078406c8f0 0x7f078406edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07a014aee0 0x7f07a014b360 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07a00c8e30 0x7f07a00c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 --2- 172.21.15.40:0/2681155563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f078c004d40 0x7f07a014e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 >> 172.21.15.40:0/2681155563 conn(0x7f07a006bfa0 msgr2=0x7f07a01541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 shutdown_connections 2024-04-25T11:38:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.688+0000 7f07a8108640 1 -- 172.21.15.40:0/2681155563 wait complete. 2024-04-25T11:38:29.689 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:38:29.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:29.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.807+0000 7f4a52368640 1 Processor -- start 2024-04-25T11:38:29.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.807+0000 7f4a52368640 1 -- start start 2024-04-25T11:38:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.808+0000 7f4a52368640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c0b6210 0x7f4a4c0c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.808+0000 7f4a52368640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a4c15f060 con 0x7f4a4c0c8690 2024-04-25T11:38:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.808+0000 7f4a52368640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a4c0d1040 con 0x7f4a4c154f50 2024-04-25T11:38:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.808+0000 7f4a52368640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c15eb20 con 0x7f4a4c0b6210 2024-04-25T11:38:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.808+0000 7f4a50905640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4a4c0c8690 0x7f4a4c0c8b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43880/0 (socket says 172.21.15.40:43880) 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.808+0000 7f4a50905640 1 -- 172.21.15.40:0/877648521 learned_addr learned my addr 172.21.15.40:0/877648521 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.808+0000 7f4a4b7fe640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c0b6210 0x7f4a4c0c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.808+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1621117040 0 0) 0x7f4a4c15f060 con 0x7f4a4c0c8690 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a38003520 con 0x7f4a4c0c8690 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 774236759 0 0) 0x7f4a4c0d1040 con 0x7f4a4c154f50 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a4c15f060 con 0x7f4a4c154f50 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3958593494 0 0) 0x7f4a38003520 con 0x7f4a4c0c8690 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c0b6210 msgr2=0x7f4a4c0c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c0b6210 0x7f4a4c0c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 >> v1:172.21.15.62:6789/0 conn(0x7f4a4c154f50 legacy=0x7f4a4c0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c0cc150 con 0x7f4a4c0c8690 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7f4a440019f0 con 0x7f4a4c0c8690 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 >> v1:172.21.15.40:6789/0 conn(0x7f4a4c0c8690 legacy=0x7f4a4c0c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a38003b70 0x7f4a38003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a38004800 0x7f4a38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c15eb20 con 0x7f4a380051c0 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3800bf30 con 0x7f4a38004800 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3800c130 con 0x7f4a38003b70 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a50905640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4bfff640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a38003b70 0x7f4a38003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a4b7fe640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a38004800 0x7f4a38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a50905640 1 -- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a38003b70 msgr2=0x7f4a38003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a50905640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a38003b70 0x7f4a38003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a50905640 1 -- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a38004800 msgr2=0x7f4a38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a50905640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a38004800 0x7f4a38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.809+0000 7f4a50905640 1 -- 172.21.15.40:0/877648521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a38003520 con 0x7f4a380051c0 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.810+0000 7f4a50905640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a3800b9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400a7c0 tx=0x7f4a38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a5550b665139c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.810+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4400a410 con 0x7f4a380051c0 2024-04-25T11:38:29.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.810+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a4402f070 con 0x7f4a380051c0 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.810+0000 7f4a4affd640 1 -- 172.21.15.40:0/877648521 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4402bc60 con 0x7f4a380051c0 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.811+0000 7f4a52368640 1 -- 172.21.15.40:0/877648521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 msgr2=0x7f4a3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.811+0000 7f4a52368640 1 --2- 172.21.15.40:0/877648521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a3800b9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400a7c0 tx=0x7f4a38005830 comp rx=0 tx=0).stop 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.811+0000 7f4a50905640 1 -- 172.21.15.40:0/877648521 reap_dead start 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.811+0000 7f4a52368640 1 -- 172.21.15.40:0/877648521 shutdown_connections 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.811+0000 7f4a52368640 1 -- 172.21.15.40:0/877648521 >> 172.21.15.40:0/877648521 conn(0x7f4a4c0d0440 msgr2=0x7f4a4c0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.811+0000 7f4a52368640 1 -- 172.21.15.40:0/877648521 shutdown_connections 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.811+0000 7f4a52368640 1 -- 172.21.15.40:0/877648521 wait complete. 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.812+0000 7f4a52368640 1 Processor -- start 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.812+0000 7f4a52368640 1 -- start start 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a52368640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a4c1f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a52368640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a4c152280 0x7f4a4c14f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a52368640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c14fa70 0x7f4a4c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a52368640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c0cc3f0 con 0x7f4a380051c0 2024-04-25T11:38:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a52368640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c15ed80 con 0x7f4a4c152280 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a52368640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c0c8170 con 0x7f4a4c14fa70 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a4c1f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a4c1f4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57434/0 (socket says 172.21.15.40:57434) 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4bfff640 1 -- 172.21.15.40:0/418677995 learned_addr learned my addr 172.21.15.40:0/418677995 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4b7fe640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a4c152280 0x7f4a4c14f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a50905640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c14fa70 0x7f4a4c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4bfff640 1 -- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c14fa70 msgr2=0x7f4a4c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4bfff640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c14fa70 0x7f4a4c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4bfff640 1 -- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a4c152280 msgr2=0x7f4a4c14f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4bfff640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a4c152280 0x7f4a4c14f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4bfff640 1 -- 172.21.15.40:0/418677995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c1564a0 con 0x7f4a380051c0 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a50905640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c14fa70 0x7f4a4c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a4bfff640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a4c1f4030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00f770 tx=0x7f4a3c00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a48ff9640 1 -- 172.21.15.40:0/418677995 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00fe80 con 0x7f4a380051c0 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a48ff9640 1 -- 172.21.15.40:0/418677995 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3c018070 con 0x7f4a380051c0 2024-04-25T11:38:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c1f4570 con 0x7f4a380051c0 2024-04-25T11:38:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a48ff9640 1 -- 172.21.15.40:0/418677995 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00d6d0 con 0x7f4a380051c0 2024-04-25T11:38:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.813+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c1556e0 con 0x7f4a380051c0 2024-04-25T11:38:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.815+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4c068bb0 con 0x7f4a380051c0 2024-04-25T11:38:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.815+0000 7f4a48ff9640 1 -- 172.21.15.40:0/418677995 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a3c00d870 con 0x7f4a380051c0 2024-04-25T11:38:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.815+0000 7f4a48ff9640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4a2c06c8f0 0x7f4a2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.815+0000 7f4a48ff9640 1 -- 172.21.15.40:0/418677995 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a3c08bb90 con 0x7f4a380051c0 2024-04-25T11:38:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.816+0000 7f4a4b7fe640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4a2c06c8f0 0x7f4a2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.816+0000 7f4a4b7fe640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4a2c06c8f0 0x7f4a2c06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c15dbf0 tx=0x7f4a34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:29.820+0000 7f4a48ff9640 1 -- 172.21.15.40:0/418677995 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a3c01a310 con 0x7f4a380051c0 2024-04-25T11:38:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.043+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a4c15baf0 con 0x7f4a380051c0 2024-04-25T11:38:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.043+0000 7f4a48ff9640 1 -- 172.21.15.40:0/418677995 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6816 (secure 0 0 0) 0x7f4a3c059f30 con 0x7f4a380051c0 2024-04-25T11:38:30.032 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:30.032 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1073487120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1073487120},{"type":"v1","addr":"172.21.15.40:6837","nonce":1073487120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6838/3333589902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3333589902},{"type":"v1","addr":"172.21.15.62:6838","nonce":3333589902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/4104143657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4104143657},{"type":"v1","addr":"172.21.15.40:6835","nonce":4104143657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6836/535060457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":535060457},{"type":"v1","addr":"172.21.15.40:6836","nonce":535060457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4294,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3758619338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3758619338},{"type":"v1","addr":"172.21.15.62:6835","nonce":3758619338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:38:25.023156+0000","modified":"2024-04-25T11:38:29.042563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.62:6839/2121129816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6837","nonce":2121129816},{"type":"v1","addr":"172.21.15.62:6839","nonce":2121129816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T11:38:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4a2c06c8f0 msgr2=0x7f4a2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4a2c06c8f0 0x7f4a2c06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c15dbf0 tx=0x7f4a34008040 comp rx=0 tx=0).stop 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 msgr2=0x7f4a4c1f4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a4c1f4030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00f770 tx=0x7f4a3c00fc30 comp rx=0 tx=0).stop 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 shutdown_connections 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4a2c06c8f0 0x7f4a2c06edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a4c14fa70 0x7f4a4c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a4c152280 0x7f4a4c14f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 --2- 172.21.15.40:0/418677995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a380051c0 0x7f4a4c1f4030 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.045+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 >> 172.21.15.40:0/418677995 conn(0x7f4a4c0d0440 msgr2=0x7f4a4c0c3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.046+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 shutdown_connections 2024-04-25T11:38:30.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.046+0000 7f4a52368640 1 -- 172.21.15.40:0/418677995 wait complete. 2024-04-25T11:38:30.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:30.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.157+0000 7fa702b7d640 1 Processor -- start 2024-04-25T11:38:30.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.157+0000 7fa702b7d640 1 -- start start 2024-04-25T11:38:30.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.158+0000 7fa702b7d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6fc0b6210 0x7fa6fc0c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.158+0000 7fa702b7d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 0x7fa6fc0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.158+0000 7fa702b7d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6fc15f060 con 0x7fa6fc154f50 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.158+0000 7fa702b7d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa6fc15eb20 con 0x7fa6fc0b6210 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.158+0000 7fa702b7d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc0c9040 con 0x7fa6fc0c8690 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.158+0000 7fa700919640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa6fc154f50 0x7fa6fc0cb2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43886/0 (socket says 172.21.15.40:43886) 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.158+0000 7fa700919640 1 -- 172.21.15.40:0/3453700608 learned_addr learned my addr 172.21.15.40:0/3453700608 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.158+0000 7fa6fbfff640 1 --2- 172.21.15.40:0/3453700608 >> v2:172.21.15.40:3300/0 conn(0x7fa6fc0b6210 0x7fa6fc0c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.158+0000 7fa70111a640 1 --2- 172.21.15.40:0/3453700608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 0x7fa6fc0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.159+0000 7fa6fbfff640 1 -- 172.21.15.40:0/3453700608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 msgr2=0x7fa6fc0c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.159+0000 7fa6fbfff640 1 --2- 172.21.15.40:0/3453700608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 0x7fa6fc0c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.159+0000 7fa6fbfff640 1 -- 172.21.15.40:0/3453700608 >> v1:172.21.15.40:6789/0 conn(0x7fa6fc154f50 legacy=0x7fa6fc0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.159+0000 7fa6fb7fe640 1 -- 172.21.15.40:0/3453700608 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 574932021 0 0) 0x7fa6fc15f060 con 0x7fa6fc154f50 2024-04-25T11:38:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.159+0000 7fa6fbfff640 1 -- 172.21.15.40:0/3453700608 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc0d1040 con 0x7fa6fc0b6210 2024-04-25T11:38:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.159+0000 7fa6fbfff640 1 --2- 172.21.15.40:0/3453700608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc0b6210 0x7fa6fc0c7a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0097a0 tx=0x7fa6ec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6f9104197598a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.159+0000 7fa6fb7fe640 1 -- 172.21.15.40:0/3453700608 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec037070 con 0x7fa6fc0b6210 2024-04-25T11:38:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa6fb7fe640 1 -- 172.21.15.40:0/3453700608 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec02ae50 con 0x7fa6fc0b6210 2024-04-25T11:38:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa6fb7fe640 1 -- 172.21.15.40:0/3453700608 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec032660 con 0x7fa6fc0b6210 2024-04-25T11:38:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa702b7d640 1 -- 172.21.15.40:0/3453700608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc0b6210 msgr2=0x7fa6fc0c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa702b7d640 1 --2- 172.21.15.40:0/3453700608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc0b6210 0x7fa6fc0c7a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0097a0 tx=0x7fa6ec02a100 comp rx=0 tx=0).stop 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa702b7d640 1 -- 172.21.15.40:0/3453700608 shutdown_connections 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa702b7d640 1 --2- 172.21.15.40:0/3453700608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 0x7fa6fc0c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa702b7d640 1 --2- 172.21.15.40:0/3453700608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc0b6210 0x7fa6fc0c7a70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa702b7d640 1 -- 172.21.15.40:0/3453700608 >> 172.21.15.40:0/3453700608 conn(0x7fa6fc0d0440 msgr2=0x7fa6fc0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa702b7d640 1 -- 172.21.15.40:0/3453700608 shutdown_connections 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.160+0000 7fa702b7d640 1 -- 172.21.15.40:0/3453700608 wait complete. 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.161+0000 7fa702b7d640 1 Processor -- start 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.161+0000 7fa702b7d640 1 -- start start 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa702b7d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 0x7fa6fc15db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa702b7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6fc154f50 0x7fa6fc1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa702b7d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc155a70 0x7fa6fc1f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa702b7d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc15ed80 con 0x7fa6fc155a70 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa702b7d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc0cc150 con 0x7fa6fc154f50 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa702b7d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc0cc3f0 con 0x7fa6fc0c8690 2024-04-25T11:38:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa70111a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc155a70 0x7fa6fc1f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa70111a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc155a70 0x7fa6fc1f0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57464/0 (socket says 172.21.15.40:57464) 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa70111a640 1 -- 172.21.15.40:0/251188876 learned_addr learned my addr 172.21.15.40:0/251188876 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa700919640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 0x7fa6fc15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa6fbfff640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6fc154f50 0x7fa6fc1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa70111a640 1 -- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 msgr2=0x7fa6fc15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa70111a640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 0x7fa6fc15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa70111a640 1 -- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6fc154f50 msgr2=0x7fa6fc1561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa70111a640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6fc154f50 0x7fa6fc1561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa70111a640 1 -- 172.21.15.40:0/251188876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc157ef0 con 0x7fa6fc155a70 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.162+0000 7fa700919640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 0x7fa6fc15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.163+0000 7fa70111a640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc155a70 0x7fa6fc1f0700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400e890 tx=0x7fa6e400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.163+0000 7fa6f97fa640 1 -- 172.21.15.40:0/251188876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4017070 con 0x7fa6fc155a70 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.163+0000 7fa6f97fa640 1 -- 172.21.15.40:0/251188876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e400c8e0 con 0x7fa6fc155a70 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.163+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc1581b0 con 0x7fa6fc155a70 2024-04-25T11:38:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.163+0000 7fa6f97fa640 1 -- 172.21.15.40:0/251188876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400ccd0 con 0x7fa6fc155a70 2024-04-25T11:38:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.163+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc15cb10 con 0x7fa6fc155a70 2024-04-25T11:38:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.164+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6fc0cbe00 con 0x7fa6fc155a70 2024-04-25T11:38:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.164+0000 7fa6f97fa640 1 -- 172.21.15.40:0/251188876 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6e401a070 con 0x7fa6fc155a70 2024-04-25T11:38:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.165+0000 7fa6f97fa640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.165+0000 7fa6f97fa640 1 -- 172.21.15.40:0/251188876 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa6e408be20 con 0x7fa6fc155a70 2024-04-25T11:38:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.165+0000 7fa700919640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.165+0000 7fa700919640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40012e0 tx=0x7fa6f4007f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.169+0000 7fa6f97fa640 1 -- 172.21.15.40:0/251188876 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6e4059b70 con 0x7fa6fc155a70 2024-04-25T11:38:30.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.358+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6fc153290 con 0x7fa6fc155a70 2024-04-25T11:38:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.359+0000 7fa6f97fa640 1 -- 172.21.15.40:0/251188876 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fa6e405de70 con 0x7fa6fc155a70 2024-04-25T11:38:30.348 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:30.348 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:26.664101+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:30.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.361+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa6dc06c8f0 msgr2=0x7fa6dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.361+0000 7fa702b7d640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40012e0 tx=0x7fa6f4007f90 comp rx=0 tx=0).stop 2024-04-25T11:38:30.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.361+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc155a70 msgr2=0x7fa6fc1f0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.361+0000 7fa702b7d640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc155a70 0x7fa6fc1f0700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400e890 tx=0x7fa6e400ed50 comp rx=0 tx=0).stop 2024-04-25T11:38:30.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.362+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 shutdown_connections 2024-04-25T11:38:30.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.362+0000 7fa702b7d640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.362+0000 7fa702b7d640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6fc155a70 0x7fa6fc1f0700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.362+0000 7fa702b7d640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6fc154f50 0x7fa6fc1561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.362+0000 7fa702b7d640 1 --2- 172.21.15.40:0/251188876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6fc0c8690 0x7fa6fc15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.362+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 >> 172.21.15.40:0/251188876 conn(0x7fa6fc0d0440 msgr2=0x7fa6fc0c9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.362+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 shutdown_connections 2024-04-25T11:38:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.362+0000 7fa702b7d640 1 -- 172.21.15.40:0/251188876 wait complete. 2024-04-25T11:38:30.361 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:38:30.361 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:38:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.473+0000 7f5619a23640 1 Processor -- start 2024-04-25T11:38:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.473+0000 7f5619a23640 1 -- start start 2024-04-25T11:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5619a23640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5614155000 0x7f56140cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5619a23640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561415b7d0 0x7f561406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5619a23640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f561406d1a0 0x7f56140cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5619a23640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f561406bc00 con 0x7f5614155000 2024-04-25T11:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5619a23640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f561406bea0 con 0x7f561406d1a0 2024-04-25T11:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5619a23640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f561406c180 con 0x7f561415b7d0 2024-04-25T11:38:30.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5612ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5614155000 0x7f56140cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5612ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5614155000 0x7f56140cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57472/0 (socket says 172.21.15.40:57472) 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5612ffd640 1 -- 172.21.15.40:0/1046947326 learned_addr learned my addr 172.21.15.40:0/1046947326 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f560bfff640 1 --2- 172.21.15.40:0/1046947326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561415b7d0 0x7f561406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f56137fe640 1 --2- 172.21.15.40:0/1046947326 >> v2:172.21.15.62:3300/0 conn(0x7f561406d1a0 0x7f56140cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5612ffd640 1 -- 172.21.15.40:0/1046947326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561415b7d0 msgr2=0x7f561406cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5612ffd640 1 --2- 172.21.15.40:0/1046947326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561415b7d0 0x7f561406cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5612ffd640 1 -- 172.21.15.40:0/1046947326 >> v2:172.21.15.62:3300/0 conn(0x7f561406d1a0 msgr2=0x7f56140cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5612ffd640 1 --2- 172.21.15.40:0/1046947326 >> v2:172.21.15.62:3300/0 conn(0x7f561406d1a0 0x7f56140cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f5612ffd640 1 -- 172.21.15.40:0/1046947326 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561406c460 con 0x7f5614155000 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.474+0000 7f560bfff640 1 --2- 172.21.15.40:0/1046947326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561415b7d0 0x7f561406cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.475+0000 7f5612ffd640 1 --2- 172.21.15.40:0/1046947326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5614155000 0x7f56140cb440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5614158980 tx=0x7f560402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f00ea36ee2a063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.475+0000 7f56127fc640 1 -- 172.21.15.40:0/1046947326 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604037070 con 0x7f5614155000 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.475+0000 7f56127fc640 1 -- 172.21.15.40:0/1046947326 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5604032440 con 0x7f5614155000 2024-04-25T11:38:30.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.475+0000 7f56127fc640 1 -- 172.21.15.40:0/1046947326 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604032810 con 0x7f5614155000 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.476+0000 7f5619a23640 1 -- 172.21.15.40:0/1046947326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5614155000 msgr2=0x7f56140cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.476+0000 7f5619a23640 1 --2- 172.21.15.40:0/1046947326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5614155000 0x7f56140cb440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5614158980 tx=0x7f560402a270 comp rx=0 tx=0).stop 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.476+0000 7f5619a23640 1 -- 172.21.15.40:0/1046947326 shutdown_connections 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.476+0000 7f5619a23640 1 --2- 172.21.15.40:0/1046947326 >> v2:172.21.15.62:3300/0 conn(0x7f561406d1a0 0x7f56140cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.476+0000 7f5619a23640 1 --2- 172.21.15.40:0/1046947326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f561415b7d0 0x7f561406cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.476+0000 7f5619a23640 1 --2- 172.21.15.40:0/1046947326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5614155000 0x7f56140cb440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.476+0000 7f5619a23640 1 -- 172.21.15.40:0/1046947326 >> 172.21.15.40:0/1046947326 conn(0x7f56140d0450 msgr2=0x7f56140d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.476+0000 7f5619a23640 1 -- 172.21.15.40:0/1046947326 shutdown_connections 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.476+0000 7f5619a23640 1 -- 172.21.15.40:0/1046947326 wait complete. 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.477+0000 7f5619a23640 1 Processor -- start 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.477+0000 7f5619a23640 1 -- start start 2024-04-25T11:38:30.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5619a23640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561406d1a0 0x7f56141f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5619a23640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5614155000 0x7f56141e5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5619a23640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561415b7d0 0x7f5614150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5619a23640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f561415bc10 con 0x7f561406d1a0 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5619a23640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56140cbb60 con 0x7f561415b7d0 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5619a23640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f56140ccc00 con 0x7f5614155000 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561406d1a0 0x7f56141f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5612ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561406d1a0 0x7f56141f4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57480/0 (socket says 172.21.15.40:57480) 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5612ffd640 1 -- 172.21.15.40:0/1633353096 learned_addr learned my addr 172.21.15.40:0/1633353096 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f56137fe640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561415b7d0 0x7f5614150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f560bfff640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5614155000 0x7f56141e5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5612ffd640 1 -- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5614155000 msgr2=0x7f56141e5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5612ffd640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5614155000 0x7f56141e5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5612ffd640 1 -- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561415b7d0 msgr2=0x7f5614150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5612ffd640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561415b7d0 0x7f5614150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5612ffd640 1 -- 172.21.15.40:0/1633353096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561415a6a0 con 0x7f561406d1a0 2024-04-25T11:38:30.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f56137fe640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561415b7d0 0x7f5614150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5612ffd640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561406d1a0 0x7f56141f4430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f560402a2b0 tx=0x7f56040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5610ff9640 1 -- 172.21.15.40:0/1633353096 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604041020 con 0x7f561406d1a0 2024-04-25T11:38:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5610ff9640 1 -- 172.21.15.40:0/1633353096 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56040306c0 con 0x7f561406d1a0 2024-04-25T11:38:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5610ff9640 1 -- 172.21.15.40:0/1633353096 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604037040 con 0x7f561406d1a0 2024-04-25T11:38:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56141e4e10 con 0x7f561406d1a0 2024-04-25T11:38:30.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.478+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56141f4cf0 con 0x7f561406d1a0 2024-04-25T11:38:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.479+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5614068bb0 con 0x7f561406d1a0 2024-04-25T11:38:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.480+0000 7f5610ff9640 1 -- 172.21.15.40:0/1633353096 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5604044020 con 0x7f561406d1a0 2024-04-25T11:38:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.480+0000 7f5610ff9640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f55f406c8f0 0x7f55f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.480+0000 7f5610ff9640 1 -- 172.21.15.40:0/1633353096 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56040ac320 con 0x7f561406d1a0 2024-04-25T11:38:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.480+0000 7f560bfff640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f55f406c8f0 0x7f55f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.481+0000 7f560bfff640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f55f406c8f0 0x7f55f406edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f56141e6e70 tx=0x7f55fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.484+0000 7f5610ff9640 1 -- 172.21.15.40:0/1633353096 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f560404b030 con 0x7f561406d1a0 2024-04-25T11:38:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.668+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56140c9800 con 0x7f55f406c8f0 2024-04-25T11:38:30.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.681+0000 7f5610ff9640 1 -- 172.21.15.40:0/1633353096 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251923 (secure 0 0 0) 0x7f56140c9800 con 0x7f55f406c8f0 2024-04-25T11:38:30.672 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:30.673 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:38:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f55f406c8f0 msgr2=0x7f55f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f55f406c8f0 0x7f55f406edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f56141e6e70 tx=0x7f55fc008040 comp rx=0 tx=0).stop 2024-04-25T11:38:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561406d1a0 msgr2=0x7f56141f4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561406d1a0 0x7f56141f4430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f560402a2b0 tx=0x7f56040046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 shutdown_connections 2024-04-25T11:38:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f55f406c8f0 0x7f55f406edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561415b7d0 0x7f5614150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5614155000 0x7f56141e5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 --2- 172.21.15.40:0/1633353096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f561406d1a0 0x7f56141f4430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 >> 172.21.15.40:0/1633353096 conn(0x7f56140d0450 msgr2=0x7f5614152f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 shutdown_connections 2024-04-25T11:38:30.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.684+0000 7f5619a23640 1 -- 172.21.15.40:0/1633353096 wait complete. 2024-04-25T11:38:30.683 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-25T11:38:28.951574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217896,"kb_used_data":1728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756824,"statfs":{"total":773094113280,"available":772870987776,"internally_reserved":0,"allocated":1769472,"data_stored":1748208,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.308040"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762879+0000","last_change":"2024-04-25T11:38:25.677572+0000","last_active":"2024-04-25T11:38:26.762879+0000","last_peered":"2024-04-25T11:38:26.762879+0000","last_clean":"2024-04-25T11:38:26.762879+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:38:26.762879+0000","last_undegraded":"2024-04-25T11:38:26.762879+0000","last_fullsized":"2024-04-25T11:38:26.762879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673367+0000","last_change":"2024-04-25T11:38:25.677437+0000","last_active":"2024-04-25T11:38:26.673367+0000","last_peered":"2024-04-25T11:38:26.673367+0000","last_clean":"2024-04-25T11:38:26.673367+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:38:26.673367+0000","last_undegraded":"2024-04-25T11:38:26.673367+0000","last_fullsized":"2024-04-25T11:38:26.673367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761847+0000","last_change":"2024-04-25T11:38:25.676553+0000","last_active":"2024-04-25T11:38:26.761847+0000","last_peered":"2024-04-25T11:38:26.761847+0000","last_clean":"2024-04-25T11:38:26.761847+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:38:26.761847+0000","last_undegraded":"2024-04-25T11:38:26.761847+0000","last_fullsized":"2024-04-25T11:38:26.761847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762862+0000","last_change":"2024-04-25T11:38:25.676518+0000","last_active":"2024-04-25T11:38:26.762862+0000","last_peered":"2024-04-25T11:38:26.762862+0000","last_clean":"2024-04-25T11:38:26.762862+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:38:26.762862+0000","last_undegraded":"2024-04-25T11:38:26.762862+0000","last_fullsized":"2024-04-25T11:38:26.762862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762062+0000","last_change":"2024-04-25T11:38:25.678524+0000","last_active":"2024-04-25T11:38:26.762062+0000","last_peered":"2024-04-25T11:38:26.762062+0000","last_clean":"2024-04-25T11:38:26.762062+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:38:26.762062+0000","last_undegraded":"2024-04-25T11:38:26.762062+0000","last_fullsized":"2024-04-25T11:38:26.762062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761774+0000","last_change":"2024-04-25T11:38:25.683983+0000","last_active":"2024-04-25T11:38:26.761774+0000","last_peered":"2024-04-25T11:38:26.761774+0000","last_clean":"2024-04-25T11:38:26.761774+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:38:26.761774+0000","last_undegraded":"2024-04-25T11:38:26.761774+0000","last_fullsized":"2024-04-25T11:38:26.761774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761720+0000","last_change":"2024-04-25T11:38:25.682904+0000","last_active":"2024-04-25T11:38:26.761720+0000","last_peered":"2024-04-25T11:38:26.761720+0000","last_clean":"2024-04-25T11:38:26.761720+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:38:26.761720+0000","last_undegraded":"2024-04-25T11:38:26.761720+0000","last_fullsized":"2024-04-25T11:38:26.761720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761274+0000","last_change":"2024-04-25T11:38:25.670312+0000","last_active":"2024-04-25T11:38:26.761274+0000","last_peered":"2024-04-25T11:38:26.761274+0000","last_clean":"2024-04-25T11:38:26.761274+0000","last_became_active":"2024-04-25T11:38:25.669988+0000","last_became_peered":"2024-04-25T11:38:25.669988+0000","last_unstale":"2024-04-25T11:38:26.761274+0000","last_undegraded":"2024-04-25T11:38:26.761274+0000","last_fullsized":"2024-04-25T11:38:26.761274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762939+0000","last_change":"2024-04-25T11:38:25.678459+0000","last_active":"2024-04-25T11:38:26.762939+0000","last_peered":"2024-04-25T11:38:26.762939+0000","last_clean":"2024-04-25T11:38:26.762939+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:38:26.762939+0000","last_undegraded":"2024-04-25T11:38:26.762939+0000","last_fullsized":"2024-04-25T11:38:26.762939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674629+0000","last_change":"2024-04-25T11:38:25.675561+0000","last_active":"2024-04-25T11:38:26.674629+0000","last_peered":"2024-04-25T11:38:26.674629+0000","last_clean":"2024-04-25T11:38:26.674629+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:38:26.674629+0000","last_undegraded":"2024-04-25T11:38:26.674629+0000","last_fullsized":"2024-04-25T11:38:26.674629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674251+0000","last_change":"2024-04-25T11:38:25.671113+0000","last_active":"2024-04-25T11:38:26.674251+0000","last_peered":"2024-04-25T11:38:26.674251+0000","last_clean":"2024-04-25T11:38:26.674251+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:38:26.674251+0000","last_undegraded":"2024-04-25T11:38:26.674251+0000","last_fullsized":"2024-04-25T11:38:26.674251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762821+0000","last_change":"2024-04-25T11:38:25.665690+0000","last_active":"2024-04-25T11:38:26.762821+0000","last_peered":"2024-04-25T11:38:26.762821+0000","last_clean":"2024-04-25T11:38:26.762821+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:38:26.762821+0000","last_undegraded":"2024-04-25T11:38:26.762821+0000","last_fullsized":"2024-04-25T11:38:26.762821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:25.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761152+0000","last_change":"2024-04-25T11:38:25.666858+0000","last_active":"2024-04-25T11:38:26.761152+0000","last_peered":"2024-04-25T11:38:26.761152+0000","last_clean":"2024-04-25T11:38:26.761152+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:38:26.761152+0000","last_undegraded":"2024-04-25T11:38:26.761152+0000","last_fullsized":"2024-04-25T11:38:26.761152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761293+0000","last_change":"2024-04-25T11:38:25.677561+0000","last_active":"2024-04-25T11:38:26.761293+0000","last_peered":"2024-04-25T11:38:26.761293+0000","last_clean":"2024-04-25T11:38:26.761293+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:38:26.761293+0000","last_undegraded":"2024-04-25T11:38:26.761293+0000","last_fullsized":"2024-04-25T11:38:26.761293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673120+0000","last_change":"2024-04-25T11:38:25.672129+0000","last_active":"2024-04-25T11:38:26.673120+0000","last_peered":"2024-04-25T11:38:26.673120+0000","last_clean":"2024-04-25T11:38:26.673120+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:38:26.673120+0000","last_undegraded":"2024-04-25T11:38:26.673120+0000","last_fullsized":"2024-04-25T11:38:26.673120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761886+0000","last_change":"2024-04-25T11:38:25.682547+0000","last_active":"2024-04-25T11:38:26.761886+0000","last_peered":"2024-04-25T11:38:26.761886+0000","last_clean":"2024-04-25T11:38:26.761886+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:38:26.761886+0000","last_undegraded":"2024-04-25T11:38:26.761886+0000","last_fullsized":"2024-04-25T11:38:26.761886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763750+0000","last_change":"2024-04-25T11:38:25.683798+0000","last_active":"2024-04-25T11:38:26.763750+0000","last_peered":"2024-04-25T11:38:26.763750+0000","last_clean":"2024-04-25T11:38:26.763750+0000","last_became_active":"2024-04-25T11:38:25.682590+0000","last_became_peered":"2024-04-25T11:38:25.682590+0000","last_unstale":"2024-04-25T11:38:26.763750+0000","last_undegraded":"2024-04-25T11:38:26.763750+0000","last_fullsized":"2024-04-25T11:38:26.763750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673869+0000","last_change":"2024-04-25T11:38:25.669650+0000","last_active":"2024-04-25T11:38:26.673869+0000","last_peered":"2024-04-25T11:38:26.673869+0000","last_clean":"2024-04-25T11:38:26.673869+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:38:26.673869+0000","last_undegraded":"2024-04-25T11:38:26.673869+0000","last_fullsized":"2024-04-25T11:38:26.673869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762614+0000","last_change":"2024-04-25T11:38:25.678238+0000","last_active":"2024-04-25T11:38:26.762614+0000","last_peered":"2024-04-25T11:38:26.762614+0000","last_clean":"2024-04-25T11:38:26.762614+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:38:26.762614+0000","last_undegraded":"2024-04-25T11:38:26.762614+0000","last_fullsized":"2024-04-25T11:38:26.762614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674167+0000","last_change":"2024-04-25T11:38:25.666736+0000","last_active":"2024-04-25T11:38:26.674167+0000","last_peered":"2024-04-25T11:38:26.674167+0000","last_clean":"2024-04-25T11:38:26.674167+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:38:26.674167+0000","last_undegraded":"2024-04-25T11:38:26.674167+0000","last_fullsized":"2024-04-25T11:38:26.674167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761661+0000","last_change":"2024-04-25T11:38:25.680587+0000","last_active":"2024-04-25T11:38:26.761661+0000","last_peered":"2024-04-25T11:38:26.761661+0000","last_clean":"2024-04-25T11:38:26.761661+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:38:26.761661+0000","last_undegraded":"2024-04-25T11:38:26.761661+0000","last_fullsized":"2024-04-25T11:38:26.761661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762157+0000","last_change":"2024-04-25T11:38:25.680464+0000","last_active":"2024-04-25T11:38:26.762157+0000","last_peered":"2024-04-25T11:38:26.762157+0000","last_clean":"2024-04-25T11:38:26.762157+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:38:26.762157+0000","last_undegraded":"2024-04-25T11:38:26.762157+0000","last_fullsized":"2024-04-25T11:38:26.762157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761201+0000","last_change":"2024-04-25T11:38:25.672189+0000","last_active":"2024-04-25T11:38:26.761201+0000","last_peered":"2024-04-25T11:38:26.761201+0000","last_clean":"2024-04-25T11:38:26.761201+0000","last_became_active":"2024-04-25T11:38:25.671920+0000","last_became_peered":"2024-04-25T11:38:25.671920+0000","last_unstale":"2024-04-25T11:38:26.761201+0000","last_undegraded":"2024-04-25T11:38:26.761201+0000","last_fullsized":"2024-04-25T11:38:26.761201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762072+0000","last_change":"2024-04-25T11:38:25.674108+0000","last_active":"2024-04-25T11:38:26.762072+0000","last_peered":"2024-04-25T11:38:26.762072+0000","last_clean":"2024-04-25T11:38:26.762072+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:38:26.762072+0000","last_undegraded":"2024-04-25T11:38:26.762072+0000","last_fullsized":"2024-04-25T11:38:26.762072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761259+0000","last_change":"2024-04-25T11:38:25.672215+0000","last_active":"2024-04-25T11:38:26.761259+0000","last_peered":"2024-04-25T11:38:26.761259+0000","last_clean":"2024-04-25T11:38:26.761259+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:38:26.761259+0000","last_undegraded":"2024-04-25T11:38:26.761259+0000","last_fullsized":"2024-04-25T11:38:26.761259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674666+0000","last_change":"2024-04-25T11:38:25.679772+0000","last_active":"2024-04-25T11:38:26.674666+0000","last_peered":"2024-04-25T11:38:26.674666+0000","last_clean":"2024-04-25T11:38:26.674666+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:38:26.674666+0000","last_undegraded":"2024-04-25T11:38:26.674666+0000","last_fullsized":"2024-04-25T11:38:26.674666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673164+0000","last_change":"2024-04-25T11:38:25.665626+0000","last_active":"2024-04-25T11:38:26.673164+0000","last_peered":"2024-04-25T11:38:26.673164+0000","last_clean":"2024-04-25T11:38:26.673164+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:38:26.673164+0000","last_undegraded":"2024-04-25T11:38:26.673164+0000","last_fullsized":"2024-04-25T11:38:26.673164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761385+0000","last_change":"2024-04-25T11:38:25.672935+0000","last_active":"2024-04-25T11:38:26.761385+0000","last_peered":"2024-04-25T11:38:26.761385+0000","last_clean":"2024-04-25T11:38:26.761385+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:38:26.761385+0000","last_undegraded":"2024-04-25T11:38:26.761385+0000","last_fullsized":"2024-04-25T11:38:26.761385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761842+0000","last_change":"2024-04-25T11:38:25.679486+0000","last_active":"2024-04-25T11:38:26.761842+0000","last_peered":"2024-04-25T11:38:26.761842+0000","last_clean":"2024-04-25T11:38:26.761842+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:38:26.761842+0000","last_undegraded":"2024-04-25T11:38:26.761842+0000","last_fullsized":"2024-04-25T11:38:26.761842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762406+0000","last_change":"2024-04-25T11:38:25.679437+0000","last_active":"2024-04-25T11:38:26.762406+0000","last_peered":"2024-04-25T11:38:26.762406+0000","last_clean":"2024-04-25T11:38:26.762406+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:38:26.762406+0000","last_undegraded":"2024-04-25T11:38:26.762406+0000","last_fullsized":"2024-04-25T11:38:26.762406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673702+0000","last_change":"2024-04-25T11:38:25.670384+0000","last_active":"2024-04-25T11:38:26.673702+0000","last_peered":"2024-04-25T11:38:26.673702+0000","last_clean":"2024-04-25T11:38:26.673702+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:38:26.673702+0000","last_undegraded":"2024-04-25T11:38:26.673702+0000","last_fullsized":"2024-04-25T11:38:26.673702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674250+0000","last_change":"2024-04-25T11:38:25.680252+0000","last_active":"2024-04-25T11:38:26.674250+0000","last_peered":"2024-04-25T11:38:26.674250+0000","last_clean":"2024-04-25T11:38:26.674250+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:38:26.674250+0000","last_undegraded":"2024-04-25T11:38:26.674250+0000","last_fullsized":"2024-04-25T11:38:26.674250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762291+0000","last_change":"2024-04-25T11:38:25.669542+0000","last_active":"2024-04-25T11:38:26.762291+0000","last_peered":"2024-04-25T11:38:26.762291+0000","last_clean":"2024-04-25T11:38:26.762291+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:38:26.762291+0000","last_undegraded":"2024-04-25T11:38:26.762291+0000","last_fullsized":"2024-04-25T11:38:26.762291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:51.929330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761399+0000","last_change":"2024-04-25T11:38:24.666197+0000","last_active":"2024-04-25T11:38:26.761399+0000","last_peered":"2024-04-25T11:38:26.761399+0000","last_clean":"2024-04-25T11:38:26.761399+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:38:26.761399+0000","last_undegraded":"2024-04-25T11:38:26.761399+0000","last_fullsized":"2024-04-25T11:38:26.761399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.198972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761974+0000","last_change":"2024-04-25T11:38:25.682931+0000","last_active":"2024-04-25T11:38:26.761974+0000","last_peered":"2024-04-25T11:38:26.761974+0000","last_clean":"2024-04-25T11:38:26.761974+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:38:26.761974+0000","last_undegraded":"2024-04-25T11:38:26.761974+0000","last_fullsized":"2024-04-25T11:38:26.761974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674162+0000","last_change":"2024-04-25T11:38:24.660888+0000","last_active":"2024-04-25T11:38:26.674162+0000","last_peered":"2024-04-25T11:38:26.674162+0000","last_clean":"2024-04-25T11:38:26.674162+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:38:26.674162+0000","last_undegraded":"2024-04-25T11:38:26.674162+0000","last_fullsized":"2024-04-25T11:38:26.674162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761513+0000","last_change":"2024-04-25T11:38:24.657815+0000","last_active":"2024-04-25T11:38:26.761513+0000","last_peered":"2024-04-25T11:38:26.761513+0000","last_clean":"2024-04-25T11:38:26.761513+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:38:26.761513+0000","last_undegraded":"2024-04-25T11:38:26.761513+0000","last_fullsized":"2024-04-25T11:38:26.761513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761437+0000","last_change":"2024-04-25T11:38:25.680969+0000","last_active":"2024-04-25T11:38:26.761437+0000","last_peered":"2024-04-25T11:38:26.761437+0000","last_clean":"2024-04-25T11:38:26.761437+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:38:26.761437+0000","last_undegraded":"2024-04-25T11:38:26.761437+0000","last_fullsized":"2024-04-25T11:38:26.761437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762182+0000","last_change":"2024-04-25T11:38:24.668899+0000","last_active":"2024-04-25T11:38:26.762182+0000","last_peered":"2024-04-25T11:38:26.762182+0000","last_clean":"2024-04-25T11:38:26.762182+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:38:26.762182+0000","last_undegraded":"2024-04-25T11:38:26.762182+0000","last_fullsized":"2024-04-25T11:38:26.762182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672975+0000","last_change":"2024-04-25T11:38:25.666511+0000","last_active":"2024-04-25T11:38:26.672975+0000","last_peered":"2024-04-25T11:38:26.672975+0000","last_clean":"2024-04-25T11:38:26.672975+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:38:26.672975+0000","last_undegraded":"2024-04-25T11:38:26.672975+0000","last_fullsized":"2024-04-25T11:38:26.672975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674478+0000","last_change":"2024-04-25T11:38:24.666735+0000","last_active":"2024-04-25T11:38:26.674478+0000","last_peered":"2024-04-25T11:38:26.674478+0000","last_clean":"2024-04-25T11:38:26.674478+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:38:26.674478+0000","last_undegraded":"2024-04-25T11:38:26.674478+0000","last_fullsized":"2024-04-25T11:38:26.674478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763586+0000","last_change":"2024-04-25T11:38:25.680065+0000","last_active":"2024-04-25T11:38:26.763586+0000","last_peered":"2024-04-25T11:38:26.763586+0000","last_clean":"2024-04-25T11:38:26.763586+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:38:26.763586+0000","last_undegraded":"2024-04-25T11:38:26.763586+0000","last_fullsized":"2024-04-25T11:38:26.763586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763667+0000","last_change":"2024-04-25T11:38:24.671566+0000","last_active":"2024-04-25T11:38:26.763667+0000","last_peered":"2024-04-25T11:38:26.763667+0000","last_clean":"2024-04-25T11:38:26.763667+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:38:26.763667+0000","last_undegraded":"2024-04-25T11:38:26.763667+0000","last_fullsized":"2024-04-25T11:38:26.763667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761910+0000","last_change":"2024-04-25T11:38:25.678075+0000","last_active":"2024-04-25T11:38:26.761910+0000","last_peered":"2024-04-25T11:38:26.761910+0000","last_clean":"2024-04-25T11:38:26.761910+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:38:26.761910+0000","last_undegraded":"2024-04-25T11:38:26.761910+0000","last_fullsized":"2024-04-25T11:38:26.761910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673019+0000","last_change":"2024-04-25T11:38:24.667282+0000","last_active":"2024-04-25T11:38:26.673019+0000","last_peered":"2024-04-25T11:38:26.673019+0000","last_clean":"2024-04-25T11:38:26.673019+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:38:26.673019+0000","last_undegraded":"2024-04-25T11:38:26.673019+0000","last_fullsized":"2024-04-25T11:38:26.673019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762833+0000","last_change":"2024-04-25T11:38:25.680417+0000","last_active":"2024-04-25T11:38:26.762833+0000","last_peered":"2024-04-25T11:38:26.762833+0000","last_clean":"2024-04-25T11:38:26.762833+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:38:26.762833+0000","last_undegraded":"2024-04-25T11:38:26.762833+0000","last_fullsized":"2024-04-25T11:38:26.762833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674339+0000","last_change":"2024-04-25T11:38:24.662562+0000","last_active":"2024-04-25T11:38:26.674339+0000","last_peered":"2024-04-25T11:38:26.674339+0000","last_clean":"2024-04-25T11:38:26.674339+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:38:26.674339+0000","last_undegraded":"2024-04-25T11:38:26.674339+0000","last_fullsized":"2024-04-25T11:38:26.674339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762135+0000","last_change":"2024-04-25T11:38:25.677518+0000","last_active":"2024-04-25T11:38:26.762135+0000","last_peered":"2024-04-25T11:38:26.762135+0000","last_clean":"2024-04-25T11:38:26.762135+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:38:26.762135+0000","last_undegraded":"2024-04-25T11:38:26.762135+0000","last_fullsized":"2024-04-25T11:38:26.762135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761935+0000","last_change":"2024-04-25T11:38:24.658960+0000","last_active":"2024-04-25T11:38:26.761935+0000","last_peered":"2024-04-25T11:38:26.761935+0000","last_clean":"2024-04-25T11:38:26.761935+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:38:26.761935+0000","last_undegraded":"2024-04-25T11:38:26.761935+0000","last_fullsized":"2024-04-25T11:38:26.761935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761438+0000","last_change":"2024-04-25T11:38:25.681355+0000","last_active":"2024-04-25T11:38:26.761438+0000","last_peered":"2024-04-25T11:38:26.761438+0000","last_clean":"2024-04-25T11:38:26.761438+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:38:26.761438+0000","last_undegraded":"2024-04-25T11:38:26.761438+0000","last_fullsized":"2024-04-25T11:38:26.761438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762505+0000","last_change":"2024-04-25T11:38:24.666747+0000","last_active":"2024-04-25T11:38:26.762505+0000","last_peered":"2024-04-25T11:38:26.762505+0000","last_clean":"2024-04-25T11:38:26.762505+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:38:26.762505+0000","last_undegraded":"2024-04-25T11:38:26.762505+0000","last_fullsized":"2024-04-25T11:38:26.762505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762221+0000","last_change":"2024-04-25T11:38:25.684128+0000","last_active":"2024-04-25T11:38:26.762221+0000","last_peered":"2024-04-25T11:38:26.762221+0000","last_clean":"2024-04-25T11:38:26.762221+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:38:26.762221+0000","last_undegraded":"2024-04-25T11:38:26.762221+0000","last_fullsized":"2024-04-25T11:38:26.762221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674047+0000","last_change":"2024-04-25T11:38:24.661103+0000","last_active":"2024-04-25T11:38:26.674047+0000","last_peered":"2024-04-25T11:38:26.674047+0000","last_clean":"2024-04-25T11:38:26.674047+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:38:26.674047+0000","last_undegraded":"2024-04-25T11:38:26.674047+0000","last_fullsized":"2024-04-25T11:38:26.674047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674380+0000","last_change":"2024-04-25T11:38:25.680490+0000","last_active":"2024-04-25T11:38:26.674380+0000","last_peered":"2024-04-25T11:38:26.674380+0000","last_clean":"2024-04-25T11:38:26.674380+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:38:26.674380+0000","last_undegraded":"2024-04-25T11:38:26.674380+0000","last_fullsized":"2024-04-25T11:38:26.674380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673688+0000","last_change":"2024-04-25T11:38:24.665238+0000","last_active":"2024-04-25T11:38:26.673688+0000","last_peered":"2024-04-25T11:38:26.673688+0000","last_clean":"2024-04-25T11:38:26.673688+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:38:26.673688+0000","last_undegraded":"2024-04-25T11:38:26.673688+0000","last_fullsized":"2024-04-25T11:38:26.673688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761773+0000","last_change":"2024-04-25T11:38:25.682669+0000","last_active":"2024-04-25T11:38:26.761773+0000","last_peered":"2024-04-25T11:38:26.761773+0000","last_clean":"2024-04-25T11:38:26.761773+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:38:26.761773+0000","last_undegraded":"2024-04-25T11:38:26.761773+0000","last_fullsized":"2024-04-25T11:38:26.761773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762021+0000","last_change":"2024-04-25T11:38:24.658666+0000","last_active":"2024-04-25T11:38:26.762021+0000","last_peered":"2024-04-25T11:38:26.762021+0000","last_clean":"2024-04-25T11:38:26.762021+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:38:26.762021+0000","last_undegraded":"2024-04-25T11:38:26.762021+0000","last_fullsized":"2024-04-25T11:38:26.762021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:24.295273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762494+0000","last_change":"2024-04-25T11:38:25.680783+0000","last_active":"2024-04-25T11:38:26.762494+0000","last_peered":"2024-04-25T11:38:26.762494+0000","last_clean":"2024-04-25T11:38:26.762494+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:38:26.762494+0000","last_undegraded":"2024-04-25T11:38:26.762494+0000","last_fullsized":"2024-04-25T11:38:26.762494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673869+0000","last_change":"2024-04-25T11:38:24.656937+0000","last_active":"2024-04-25T11:38:26.673869+0000","last_peered":"2024-04-25T11:38:26.673869+0000","last_clean":"2024-04-25T11:38:26.673869+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:38:26.673869+0000","last_undegraded":"2024-04-25T11:38:26.673869+0000","last_fullsized":"2024-04-25T11:38:26.673869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763039+0000","last_change":"2024-04-25T11:38:25.681307+0000","last_active":"2024-04-25T11:38:26.763039+0000","last_peered":"2024-04-25T11:38:26.763039+0000","last_clean":"2024-04-25T11:38:26.763039+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:38:26.763039+0000","last_undegraded":"2024-04-25T11:38:26.763039+0000","last_fullsized":"2024-04-25T11:38:26.763039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:17.678183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673129+0000","last_change":"2024-04-25T11:38:24.663247+0000","last_active":"2024-04-25T11:38:26.673129+0000","last_peered":"2024-04-25T11:38:26.673129+0000","last_clean":"2024-04-25T11:38:26.673129+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:38:26.673129+0000","last_undegraded":"2024-04-25T11:38:26.673129+0000","last_fullsized":"2024-04-25T11:38:26.673129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763176+0000","last_change":"2024-04-25T11:38:25.676409+0000","last_active":"2024-04-25T11:38:26.763176+0000","last_peered":"2024-04-25T11:38:26.763176+0000","last_clean":"2024-04-25T11:38:26.763176+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.763176+0000","last_undegraded":"2024-04-25T11:38:26.763176+0000","last_fullsized":"2024-04-25T11:38:26.763176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673831+0000","last_change":"2024-04-25T11:38:24.666800+0000","last_active":"2024-04-25T11:38:26.673831+0000","last_peered":"2024-04-25T11:38:26.673831+0000","last_clean":"2024-04-25T11:38:26.673831+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:38:26.673831+0000","last_undegraded":"2024-04-25T11:38:26.673831+0000","last_fullsized":"2024-04-25T11:38:26.673831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762286+0000","last_change":"2024-04-25T11:38:25.678136+0000","last_active":"2024-04-25T11:38:26.762286+0000","last_peered":"2024-04-25T11:38:26.762286+0000","last_clean":"2024-04-25T11:38:26.762286+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:38:26.762286+0000","last_undegraded":"2024-04-25T11:38:26.762286+0000","last_fullsized":"2024-04-25T11:38:26.762286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761547+0000","last_change":"2024-04-25T11:38:20.865911+0000","last_active":"2024-04-25T11:38:26.761547+0000","last_peered":"2024-04-25T11:38:26.761547+0000","last_clean":"2024-04-25T11:38:26.761547+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:38:26.761547+0000","last_undegraded":"2024-04-25T11:38:26.761547+0000","last_fullsized":"2024-04-25T11:38:26.761547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_clean_scrub_stamp":"2024-04-25T11:38:19.606342+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:58.126842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762328+0000","last_change":"2024-04-25T11:38:24.671283+0000","last_active":"2024-04-25T11:38:26.762328+0000","last_peered":"2024-04-25T11:38:26.762328+0000","last_clean":"2024-04-25T11:38:26.762328+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:38:26.762328+0000","last_undegraded":"2024-04-25T11:38:26.762328+0000","last_fullsized":"2024-04-25T11:38:26.762328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761567+0000","last_change":"2024-04-25T11:38:25.675002+0000","last_active":"2024-04-25T11:38:26.761567+0000","last_peered":"2024-04-25T11:38:26.761567+0000","last_clean":"2024-04-25T11:38:26.761567+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:38:26.761567+0000","last_undegraded":"2024-04-25T11:38:26.761567+0000","last_fullsized":"2024-04-25T11:38:26.761567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761489+0000","last_change":"2024-04-25T11:38:24.671213+0000","last_active":"2024-04-25T11:38:26.761489+0000","last_peered":"2024-04-25T11:38:26.761489+0000","last_clean":"2024-04-25T11:38:26.761489+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:38:26.761489+0000","last_undegraded":"2024-04-25T11:38:26.761489+0000","last_fullsized":"2024-04-25T11:38:26.761489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761052+0000","last_change":"2024-04-25T11:38:25.673353+0000","last_active":"2024-04-25T11:38:26.761052+0000","last_peered":"2024-04-25T11:38:26.761052+0000","last_clean":"2024-04-25T11:38:26.761052+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:38:26.761052+0000","last_undegraded":"2024-04-25T11:38:26.761052+0000","last_fullsized":"2024-04-25T11:38:26.761052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:48.553601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673682+0000","last_change":"2024-04-25T11:38:24.650599+0000","last_active":"2024-04-25T11:38:26.673682+0000","last_peered":"2024-04-25T11:38:26.673682+0000","last_clean":"2024-04-25T11:38:26.673682+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:38:26.673682+0000","last_undegraded":"2024-04-25T11:38:26.673682+0000","last_fullsized":"2024-04-25T11:38:26.673682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672982+0000","last_change":"2024-04-25T11:38:25.668956+0000","last_active":"2024-04-25T11:38:26.672982+0000","last_peered":"2024-04-25T11:38:26.672982+0000","last_clean":"2024-04-25T11:38:26.672982+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:38:26.672982+0000","last_undegraded":"2024-04-25T11:38:26.672982+0000","last_fullsized":"2024-04-25T11:38:26.672982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762612+0000","last_change":"2024-04-25T11:38:24.665496+0000","last_active":"2024-04-25T11:38:26.762612+0000","last_peered":"2024-04-25T11:38:26.762612+0000","last_clean":"2024-04-25T11:38:26.762612+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:38:26.762612+0000","last_undegraded":"2024-04-25T11:38:26.762612+0000","last_fullsized":"2024-04-25T11:38:26.762612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762812+0000","last_change":"2024-04-25T11:38:25.681800+0000","last_active":"2024-04-25T11:38:26.762812+0000","last_peered":"2024-04-25T11:38:26.762812+0000","last_clean":"2024-04-25T11:38:26.762812+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:38:26.762812+0000","last_undegraded":"2024-04-25T11:38:26.762812+0000","last_fullsized":"2024-04-25T11:38:26.762812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:39.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673142+0000","last_change":"2024-04-25T11:38:24.659792+0000","last_active":"2024-04-25T11:38:26.673142+0000","last_peered":"2024-04-25T11:38:26.673142+0000","last_clean":"2024-04-25T11:38:26.673142+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:38:26.673142+0000","last_undegraded":"2024-04-25T11:38:26.673142+0000","last_fullsized":"2024-04-25T11:38:26.673142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763580+0000","last_change":"2024-04-25T11:38:25.673411+0000","last_active":"2024-04-25T11:38:26.763580+0000","last_peered":"2024-04-25T11:38:26.763580+0000","last_clean":"2024-04-25T11:38:26.763580+0000","last_became_active":"2024-04-25T11:38:25.673116+0000","last_became_peered":"2024-04-25T11:38:25.673116+0000","last_unstale":"2024-04-25T11:38:26.763580+0000","last_undegraded":"2024-04-25T11:38:26.763580+0000","last_fullsized":"2024-04-25T11:38:26.763580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762671+0000","last_change":"2024-04-25T11:38:24.671412+0000","last_active":"2024-04-25T11:38:26.762671+0000","last_peered":"2024-04-25T11:38:26.762671+0000","last_clean":"2024-04-25T11:38:26.762671+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:38:26.762671+0000","last_undegraded":"2024-04-25T11:38:26.762671+0000","last_fullsized":"2024-04-25T11:38:26.762671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:34.607841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673048+0000","last_change":"2024-04-25T11:38:25.668959+0000","last_active":"2024-04-25T11:38:26.673048+0000","last_peered":"2024-04-25T11:38:26.673048+0000","last_clean":"2024-04-25T11:38:26.673048+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:38:26.673048+0000","last_undegraded":"2024-04-25T11:38:26.673048+0000","last_fullsized":"2024-04-25T11:38:26.673048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673459+0000","last_change":"2024-04-25T11:38:24.664146+0000","last_active":"2024-04-25T11:38:26.673459+0000","last_peered":"2024-04-25T11:38:26.673459+0000","last_clean":"2024-04-25T11:38:26.673459+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:38:26.673459+0000","last_undegraded":"2024-04-25T11:38:26.673459+0000","last_fullsized":"2024-04-25T11:38:26.673459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761339+0000","last_change":"2024-04-25T11:38:25.680105+0000","last_active":"2024-04-25T11:38:26.761339+0000","last_peered":"2024-04-25T11:38:26.761339+0000","last_clean":"2024-04-25T11:38:26.761339+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:38:26.761339+0000","last_undegraded":"2024-04-25T11:38:26.761339+0000","last_fullsized":"2024-04-25T11:38:26.761339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674386+0000","last_change":"2024-04-25T11:38:24.668369+0000","last_active":"2024-04-25T11:38:26.674386+0000","last_peered":"2024-04-25T11:38:26.674386+0000","last_clean":"2024-04-25T11:38:26.674386+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:38:26.674386+0000","last_undegraded":"2024-04-25T11:38:26.674386+0000","last_fullsized":"2024-04-25T11:38:26.674386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761723+0000","last_change":"2024-04-25T11:38:25.676446+0000","last_active":"2024-04-25T11:38:26.761723+0000","last_peered":"2024-04-25T11:38:26.761723+0000","last_clean":"2024-04-25T11:38:26.761723+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.761723+0000","last_undegraded":"2024-04-25T11:38:26.761723+0000","last_fullsized":"2024-04-25T11:38:26.761723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761379+0000","last_change":"2024-04-25T11:38:24.671951+0000","last_active":"2024-04-25T11:38:26.761379+0000","last_peered":"2024-04-25T11:38:26.761379+0000","last_clean":"2024-04-25T11:38:26.761379+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:38:26.761379+0000","last_undegraded":"2024-04-25T11:38:26.761379+0000","last_fullsized":"2024-04-25T11:38:26.761379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761852+0000","last_change":"2024-04-25T11:38:25.669474+0000","last_active":"2024-04-25T11:38:26.761852+0000","last_peered":"2024-04-25T11:38:26.761852+0000","last_clean":"2024-04-25T11:38:26.761852+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:38:26.761852+0000","last_undegraded":"2024-04-25T11:38:26.761852+0000","last_fullsized":"2024-04-25T11:38:26.761852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:38.359208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673212+0000","last_change":"2024-04-25T11:38:24.668689+0000","last_active":"2024-04-25T11:38:26.673212+0000","last_peered":"2024-04-25T11:38:26.673212+0000","last_clean":"2024-04-25T11:38:26.673212+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:38:26.673212+0000","last_undegraded":"2024-04-25T11:38:26.673212+0000","last_fullsized":"2024-04-25T11:38:26.673212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761421+0000","last_change":"2024-04-25T11:38:25.668088+0000","last_active":"2024-04-25T11:38:26.761421+0000","last_peered":"2024-04-25T11:38:26.761421+0000","last_clean":"2024-04-25T11:38:26.761421+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:38:26.761421+0000","last_undegraded":"2024-04-25T11:38:26.761421+0000","last_fullsized":"2024-04-25T11:38:26.761421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673373+0000","last_change":"2024-04-25T11:38:24.662913+0000","last_active":"2024-04-25T11:38:26.673373+0000","last_peered":"2024-04-25T11:38:26.673373+0000","last_clean":"2024-04-25T11:38:26.673373+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:38:26.673373+0000","last_undegraded":"2024-04-25T11:38:26.673373+0000","last_fullsized":"2024-04-25T11:38:26.673373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761308+0000","last_change":"2024-04-25T11:38:25.680244+0000","last_active":"2024-04-25T11:38:26.761308+0000","last_peered":"2024-04-25T11:38:26.761308+0000","last_clean":"2024-04-25T11:38:26.761308+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:38:26.761308+0000","last_undegraded":"2024-04-25T11:38:26.761308+0000","last_fullsized":"2024-04-25T11:38:26.761308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762404+0000","last_change":"2024-04-25T11:38:24.669196+0000","last_active":"2024-04-25T11:38:26.762404+0000","last_peered":"2024-04-25T11:38:26.762404+0000","last_clean":"2024-04-25T11:38:26.762404+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:38:26.762404+0000","last_undegraded":"2024-04-25T11:38:26.762404+0000","last_fullsized":"2024-04-25T11:38:26.762404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.791358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674198+0000","last_change":"2024-04-25T11:38:25.668400+0000","last_active":"2024-04-25T11:38:26.674198+0000","last_peered":"2024-04-25T11:38:26.674198+0000","last_clean":"2024-04-25T11:38:26.674198+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:38:26.674198+0000","last_undegraded":"2024-04-25T11:38:26.674198+0000","last_fullsized":"2024-04-25T11:38:26.674198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763520+0000","last_change":"2024-04-25T11:38:25.683663+0000","last_active":"2024-04-25T11:38:26.763520+0000","last_peered":"2024-04-25T11:38:26.763520+0000","last_clean":"2024-04-25T11:38:26.763520+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:38:26.763520+0000","last_undegraded":"2024-04-25T11:38:26.763520+0000","last_fullsized":"2024-04-25T11:38:26.763520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761944+0000","last_change":"2024-04-25T11:38:24.670671+0000","last_active":"2024-04-25T11:38:26.761944+0000","last_peered":"2024-04-25T11:38:26.761944+0000","last_clean":"2024-04-25T11:38:26.761944+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:38:26.761944+0000","last_undegraded":"2024-04-25T11:38:26.761944+0000","last_fullsized":"2024-04-25T11:38:26.761944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763017+0000","last_change":"2024-04-25T11:38:25.681439+0000","last_active":"2024-04-25T11:38:26.763017+0000","last_peered":"2024-04-25T11:38:26.763017+0000","last_clean":"2024-04-25T11:38:26.763017+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:38:26.763017+0000","last_undegraded":"2024-04-25T11:38:26.763017+0000","last_fullsized":"2024-04-25T11:38:26.763017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762567+0000","last_change":"2024-04-25T11:38:24.669485+0000","last_active":"2024-04-25T11:38:26.762567+0000","last_peered":"2024-04-25T11:38:26.762567+0000","last_clean":"2024-04-25T11:38:26.762567+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:38:26.762567+0000","last_undegraded":"2024-04-25T11:38:26.762567+0000","last_fullsized":"2024-04-25T11:38:26.762567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761766+0000","last_change":"2024-04-25T11:38:25.684415+0000","last_active":"2024-04-25T11:38:26.761766+0000","last_peered":"2024-04-25T11:38:26.761766+0000","last_clean":"2024-04-25T11:38:26.761766+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:38:26.761766+0000","last_undegraded":"2024-04-25T11:38:26.761766+0000","last_fullsized":"2024-04-25T11:38:26.761766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761477+0000","last_change":"2024-04-25T11:38:24.658741+0000","last_active":"2024-04-25T11:38:26.761477+0000","last_peered":"2024-04-25T11:38:26.761477+0000","last_clean":"2024-04-25T11:38:26.761477+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:38:26.761477+0000","last_undegraded":"2024-04-25T11:38:26.761477+0000","last_fullsized":"2024-04-25T11:38:26.761477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761168+0000","last_change":"2024-04-25T11:38:25.667505+0000","last_active":"2024-04-25T11:38:26.761168+0000","last_peered":"2024-04-25T11:38:26.761168+0000","last_clean":"2024-04-25T11:38:26.761168+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:38:26.761168+0000","last_undegraded":"2024-04-25T11:38:26.761168+0000","last_fullsized":"2024-04-25T11:38:26.761168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761936+0000","last_change":"2024-04-25T11:38:24.669215+0000","last_active":"2024-04-25T11:38:26.761936+0000","last_peered":"2024-04-25T11:38:26.761936+0000","last_clean":"2024-04-25T11:38:26.761936+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:38:26.761936+0000","last_undegraded":"2024-04-25T11:38:26.761936+0000","last_fullsized":"2024-04-25T11:38:26.761936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":661302,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":661302,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:38:30.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:38:30.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.837+0000 7faeb14cf640 1 Processor -- start 2024-04-25T11:38:30.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.838+0000 7faeb14cf640 1 -- start start 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.838+0000 7faeb14cf640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faeac154f50 0x7faeac0c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.838+0000 7faeb14cf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faeac0b6210 0x7faeac0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.838+0000 7faeb14cf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac15f060 con 0x7faeac0cb7e0 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.838+0000 7faeb14cf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7faeac15eb20 con 0x7faeac0b6210 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.838+0000 7faeb14cf640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7faeac0d1040 con 0x7faeac154f50 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.838+0000 7faeaa7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faeac0b6210 0x7faeac0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.838+0000 7faeab7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faeac0cb7e0 0x7faeac0c8690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43888/0 (socket says 172.21.15.40:43888) 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.839+0000 7faeab7fe640 1 -- 172.21.15.40:0/3195222879 learned_addr learned my addr 172.21.15.40:0/3195222879 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.838+0000 7faeaa7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faeac0b6210 0x7faeac0cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57490/0 (socket says 172.21.15.40:57490) 2024-04-25T11:38:30.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.839+0000 7faeaaffd640 1 --2- 172.21.15.40:0/3195222879 >> v2:172.21.15.62:3300/0 conn(0x7faeac154f50 0x7faeac0c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.839+0000 7faeaa7fc640 1 -- 172.21.15.40:0/3195222879 >> v2:172.21.15.62:3300/0 conn(0x7faeac154f50 msgr2=0x7faeac0c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.839+0000 7faeaa7fc640 1 --2- 172.21.15.40:0/3195222879 >> v2:172.21.15.62:3300/0 conn(0x7faeac154f50 0x7faeac0c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.839+0000 7faeaa7fc640 1 -- 172.21.15.40:0/3195222879 >> v1:172.21.15.40:6789/0 conn(0x7faeac0cb7e0 legacy=0x7faeac0c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.839+0000 7faea9ffb640 1 -- 172.21.15.40:0/3195222879 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599920027 0 0) 0x7faeac15f060 con 0x7faeac0cb7e0 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.839+0000 7faeaa7fc640 1 -- 172.21.15.40:0/3195222879 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac0cc150 con 0x7faeac0b6210 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.839+0000 7faeaaffd640 1 --2- 172.21.15.40:0/3195222879 >> v2:172.21.15.62:3300/0 conn(0x7faeac154f50 0x7faeac0c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faeaa7fc640 1 --2- 172.21.15.40:0/3195222879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac0b6210 0x7faeac0cb2a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faeac08d680 tx=0x7faea002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ad2de74a8f8711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faea9ffb640 1 -- 172.21.15.40:0/3195222879 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0037070 con 0x7faeac0b6210 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faea9ffb640 1 -- 172.21.15.40:0/3195222879 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea0032440 con 0x7faeac0b6210 2024-04-25T11:38:30.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faea9ffb640 1 -- 172.21.15.40:0/3195222879 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea002ae60 con 0x7faeac0b6210 2024-04-25T11:38:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faeb14cf640 1 -- 172.21.15.40:0/3195222879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac0b6210 msgr2=0x7faeac0cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faeb14cf640 1 --2- 172.21.15.40:0/3195222879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac0b6210 0x7faeac0cb2a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faeac08d680 tx=0x7faea002a110 comp rx=0 tx=0).stop 2024-04-25T11:38:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faeb14cf640 1 -- 172.21.15.40:0/3195222879 shutdown_connections 2024-04-25T11:38:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faeb14cf640 1 --2- 172.21.15.40:0/3195222879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac0b6210 0x7faeac0cb2a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faeb14cf640 1 --2- 172.21.15.40:0/3195222879 >> v2:172.21.15.62:3300/0 conn(0x7faeac154f50 0x7faeac0c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faeb14cf640 1 -- 172.21.15.40:0/3195222879 >> 172.21.15.40:0/3195222879 conn(0x7faeac0d0440 msgr2=0x7faeac0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faeb14cf640 1 -- 172.21.15.40:0/3195222879 shutdown_connections 2024-04-25T11:38:30.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.840+0000 7faeb14cf640 1 -- 172.21.15.40:0/3195222879 wait complete. 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeb14cf640 1 Processor -- start 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeb14cf640 1 -- start start 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeb14cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeac0cb7e0 0x7faeac0c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeb14cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac154f50 0x7faeac1f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeb14cf640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac1f40c0 0x7faeac14a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeb14cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faeac0cc3f0 con 0x7faeac154f50 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeb14cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faeac15ed80 con 0x7faeac0cb7e0 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeb14cf640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faeac0c8170 con 0x7faeac1f40c0 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeaa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac154f50 0x7faeac1f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeaa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac154f50 0x7faeac1f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57502/0 (socket says 172.21.15.40:57502) 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.842+0000 7faeaa7fc640 1 -- 172.21.15.40:0/3136455765 learned_addr learned my addr 172.21.15.40:0/3136455765 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeaaffd640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeac0cb7e0 0x7faeac0c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeab7fe640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac1f40c0 0x7faeac14a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeaa7fc640 1 -- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac1f40c0 msgr2=0x7faeac14a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeaa7fc640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac1f40c0 0x7faeac14a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeaa7fc640 1 -- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeac0cb7e0 msgr2=0x7faeac0c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeaa7fc640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeac0cb7e0 0x7faeac0c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeaa7fc640 1 -- 172.21.15.40:0/3136455765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac0c6ce0 con 0x7faeac154f50 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeab7fe640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac1f40c0 0x7faeac14a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeaa7fc640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac154f50 0x7faeac1f3b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faea0009890 tx=0x7faea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7fae937fe640 1 -- 172.21.15.40:0/3136455765 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0037040 con 0x7faeac154f50 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7fae937fe640 1 -- 172.21.15.40:0/3136455765 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea0042070 con 0x7faeac154f50 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac158300 con 0x7faeac154f50 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7fae937fe640 1 -- 172.21.15.40:0/3136455765 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0030a50 con 0x7faeac154f50 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.843+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac1f4540 con 0x7faeac154f50 2024-04-25T11:38:30.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.844+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeac153ab0 con 0x7faeac154f50 2024-04-25T11:38:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.844+0000 7fae937fe640 1 -- 172.21.15.40:0/3136455765 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faea0030bf0 con 0x7faeac154f50 2024-04-25T11:38:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.845+0000 7fae937fe640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae9806c8f0 0x7fae9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.845+0000 7fae937fe640 1 -- 172.21.15.40:0/3136455765 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faea00ac610 con 0x7faeac154f50 2024-04-25T11:38:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.845+0000 7faeaaffd640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae9806c8f0 0x7fae9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.845+0000 7faeaaffd640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae9806c8f0 0x7fae9806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00a990 tx=0x7fae9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:30.849+0000 7fae937fe640 1 -- 172.21.15.40:0/3136455765 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faeac153ab0 con 0x7faeac154f50 2024-04-25T11:38:31.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.031+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faeac1585a0 con 0x7fae9806c8f0 2024-04-25T11:38:31.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.038+0000 7fae937fe640 1 -- 172.21.15.40:0/3136455765 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251923 (secure 0 0 0) 0x7faeac1585a0 con 0x7fae9806c8f0 2024-04-25T11:38:31.029 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:31.030 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae9806c8f0 msgr2=0x7fae9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae9806c8f0 0x7fae9806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00a990 tx=0x7fae9c008040 comp rx=0 tx=0).stop 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac154f50 msgr2=0x7faeac1f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac154f50 0x7faeac1f3b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faea0009890 tx=0x7faea00046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 shutdown_connections 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae9806c8f0 0x7fae9806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faeac1f40c0 0x7faeac14a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faeac154f50 0x7faeac1f3b80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 --2- 172.21.15.40:0/3136455765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeac0cb7e0 0x7faeac0c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 >> 172.21.15.40:0/3136455765 conn(0x7faeac0d0440 msgr2=0x7faeac06c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 shutdown_connections 2024-04-25T11:38:31.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.041+0000 7faeb14cf640 1 -- 172.21.15.40:0/3136455765 wait complete. 2024-04-25T11:38:31.040 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-25T11:38:30.952173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217896,"kb_used_data":1728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756824,"statfs":{"total":773094113280,"available":772870987776,"internally_reserved":0,"allocated":1769472,"data_stored":1748208,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.308176"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762879+0000","last_change":"2024-04-25T11:38:25.677572+0000","last_active":"2024-04-25T11:38:26.762879+0000","last_peered":"2024-04-25T11:38:26.762879+0000","last_clean":"2024-04-25T11:38:26.762879+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:38:26.762879+0000","last_undegraded":"2024-04-25T11:38:26.762879+0000","last_fullsized":"2024-04-25T11:38:26.762879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673367+0000","last_change":"2024-04-25T11:38:25.677437+0000","last_active":"2024-04-25T11:38:26.673367+0000","last_peered":"2024-04-25T11:38:26.673367+0000","last_clean":"2024-04-25T11:38:26.673367+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:38:26.673367+0000","last_undegraded":"2024-04-25T11:38:26.673367+0000","last_fullsized":"2024-04-25T11:38:26.673367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761847+0000","last_change":"2024-04-25T11:38:25.676553+0000","last_active":"2024-04-25T11:38:26.761847+0000","last_peered":"2024-04-25T11:38:26.761847+0000","last_clean":"2024-04-25T11:38:26.761847+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:38:26.761847+0000","last_undegraded":"2024-04-25T11:38:26.761847+0000","last_fullsized":"2024-04-25T11:38:26.761847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762862+0000","last_change":"2024-04-25T11:38:25.676518+0000","last_active":"2024-04-25T11:38:26.762862+0000","last_peered":"2024-04-25T11:38:26.762862+0000","last_clean":"2024-04-25T11:38:26.762862+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:38:26.762862+0000","last_undegraded":"2024-04-25T11:38:26.762862+0000","last_fullsized":"2024-04-25T11:38:26.762862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762062+0000","last_change":"2024-04-25T11:38:25.678524+0000","last_active":"2024-04-25T11:38:26.762062+0000","last_peered":"2024-04-25T11:38:26.762062+0000","last_clean":"2024-04-25T11:38:26.762062+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:38:26.762062+0000","last_undegraded":"2024-04-25T11:38:26.762062+0000","last_fullsized":"2024-04-25T11:38:26.762062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761774+0000","last_change":"2024-04-25T11:38:25.683983+0000","last_active":"2024-04-25T11:38:26.761774+0000","last_peered":"2024-04-25T11:38:26.761774+0000","last_clean":"2024-04-25T11:38:26.761774+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:38:26.761774+0000","last_undegraded":"2024-04-25T11:38:26.761774+0000","last_fullsized":"2024-04-25T11:38:26.761774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761720+0000","last_change":"2024-04-25T11:38:25.682904+0000","last_active":"2024-04-25T11:38:26.761720+0000","last_peered":"2024-04-25T11:38:26.761720+0000","last_clean":"2024-04-25T11:38:26.761720+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:38:26.761720+0000","last_undegraded":"2024-04-25T11:38:26.761720+0000","last_fullsized":"2024-04-25T11:38:26.761720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761274+0000","last_change":"2024-04-25T11:38:25.670312+0000","last_active":"2024-04-25T11:38:26.761274+0000","last_peered":"2024-04-25T11:38:26.761274+0000","last_clean":"2024-04-25T11:38:26.761274+0000","last_became_active":"2024-04-25T11:38:25.669988+0000","last_became_peered":"2024-04-25T11:38:25.669988+0000","last_unstale":"2024-04-25T11:38:26.761274+0000","last_undegraded":"2024-04-25T11:38:26.761274+0000","last_fullsized":"2024-04-25T11:38:26.761274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762939+0000","last_change":"2024-04-25T11:38:25.678459+0000","last_active":"2024-04-25T11:38:26.762939+0000","last_peered":"2024-04-25T11:38:26.762939+0000","last_clean":"2024-04-25T11:38:26.762939+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:38:26.762939+0000","last_undegraded":"2024-04-25T11:38:26.762939+0000","last_fullsized":"2024-04-25T11:38:26.762939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674629+0000","last_change":"2024-04-25T11:38:25.675561+0000","last_active":"2024-04-25T11:38:26.674629+0000","last_peered":"2024-04-25T11:38:26.674629+0000","last_clean":"2024-04-25T11:38:26.674629+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:38:26.674629+0000","last_undegraded":"2024-04-25T11:38:26.674629+0000","last_fullsized":"2024-04-25T11:38:26.674629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674251+0000","last_change":"2024-04-25T11:38:25.671113+0000","last_active":"2024-04-25T11:38:26.674251+0000","last_peered":"2024-04-25T11:38:26.674251+0000","last_clean":"2024-04-25T11:38:26.674251+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:38:26.674251+0000","last_undegraded":"2024-04-25T11:38:26.674251+0000","last_fullsized":"2024-04-25T11:38:26.674251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762821+0000","last_change":"2024-04-25T11:38:25.665690+0000","last_active":"2024-04-25T11:38:26.762821+0000","last_peered":"2024-04-25T11:38:26.762821+0000","last_clean":"2024-04-25T11:38:26.762821+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:38:26.762821+0000","last_undegraded":"2024-04-25T11:38:26.762821+0000","last_fullsized":"2024-04-25T11:38:26.762821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:25.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761152+0000","last_change":"2024-04-25T11:38:25.666858+0000","last_active":"2024-04-25T11:38:26.761152+0000","last_peered":"2024-04-25T11:38:26.761152+0000","last_clean":"2024-04-25T11:38:26.761152+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:38:26.761152+0000","last_undegraded":"2024-04-25T11:38:26.761152+0000","last_fullsized":"2024-04-25T11:38:26.761152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761293+0000","last_change":"2024-04-25T11:38:25.677561+0000","last_active":"2024-04-25T11:38:26.761293+0000","last_peered":"2024-04-25T11:38:26.761293+0000","last_clean":"2024-04-25T11:38:26.761293+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:38:26.761293+0000","last_undegraded":"2024-04-25T11:38:26.761293+0000","last_fullsized":"2024-04-25T11:38:26.761293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673120+0000","last_change":"2024-04-25T11:38:25.672129+0000","last_active":"2024-04-25T11:38:26.673120+0000","last_peered":"2024-04-25T11:38:26.673120+0000","last_clean":"2024-04-25T11:38:26.673120+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:38:26.673120+0000","last_undegraded":"2024-04-25T11:38:26.673120+0000","last_fullsized":"2024-04-25T11:38:26.673120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761886+0000","last_change":"2024-04-25T11:38:25.682547+0000","last_active":"2024-04-25T11:38:26.761886+0000","last_peered":"2024-04-25T11:38:26.761886+0000","last_clean":"2024-04-25T11:38:26.761886+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:38:26.761886+0000","last_undegraded":"2024-04-25T11:38:26.761886+0000","last_fullsized":"2024-04-25T11:38:26.761886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763750+0000","last_change":"2024-04-25T11:38:25.683798+0000","last_active":"2024-04-25T11:38:26.763750+0000","last_peered":"2024-04-25T11:38:26.763750+0000","last_clean":"2024-04-25T11:38:26.763750+0000","last_became_active":"2024-04-25T11:38:25.682590+0000","last_became_peered":"2024-04-25T11:38:25.682590+0000","last_unstale":"2024-04-25T11:38:26.763750+0000","last_undegraded":"2024-04-25T11:38:26.763750+0000","last_fullsized":"2024-04-25T11:38:26.763750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673869+0000","last_change":"2024-04-25T11:38:25.669650+0000","last_active":"2024-04-25T11:38:26.673869+0000","last_peered":"2024-04-25T11:38:26.673869+0000","last_clean":"2024-04-25T11:38:26.673869+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:38:26.673869+0000","last_undegraded":"2024-04-25T11:38:26.673869+0000","last_fullsized":"2024-04-25T11:38:26.673869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762614+0000","last_change":"2024-04-25T11:38:25.678238+0000","last_active":"2024-04-25T11:38:26.762614+0000","last_peered":"2024-04-25T11:38:26.762614+0000","last_clean":"2024-04-25T11:38:26.762614+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:38:26.762614+0000","last_undegraded":"2024-04-25T11:38:26.762614+0000","last_fullsized":"2024-04-25T11:38:26.762614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674167+0000","last_change":"2024-04-25T11:38:25.666736+0000","last_active":"2024-04-25T11:38:26.674167+0000","last_peered":"2024-04-25T11:38:26.674167+0000","last_clean":"2024-04-25T11:38:26.674167+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:38:26.674167+0000","last_undegraded":"2024-04-25T11:38:26.674167+0000","last_fullsized":"2024-04-25T11:38:26.674167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761661+0000","last_change":"2024-04-25T11:38:25.680587+0000","last_active":"2024-04-25T11:38:26.761661+0000","last_peered":"2024-04-25T11:38:26.761661+0000","last_clean":"2024-04-25T11:38:26.761661+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:38:26.761661+0000","last_undegraded":"2024-04-25T11:38:26.761661+0000","last_fullsized":"2024-04-25T11:38:26.761661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762157+0000","last_change":"2024-04-25T11:38:25.680464+0000","last_active":"2024-04-25T11:38:26.762157+0000","last_peered":"2024-04-25T11:38:26.762157+0000","last_clean":"2024-04-25T11:38:26.762157+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:38:26.762157+0000","last_undegraded":"2024-04-25T11:38:26.762157+0000","last_fullsized":"2024-04-25T11:38:26.762157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761201+0000","last_change":"2024-04-25T11:38:25.672189+0000","last_active":"2024-04-25T11:38:26.761201+0000","last_peered":"2024-04-25T11:38:26.761201+0000","last_clean":"2024-04-25T11:38:26.761201+0000","last_became_active":"2024-04-25T11:38:25.671920+0000","last_became_peered":"2024-04-25T11:38:25.671920+0000","last_unstale":"2024-04-25T11:38:26.761201+0000","last_undegraded":"2024-04-25T11:38:26.761201+0000","last_fullsized":"2024-04-25T11:38:26.761201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762072+0000","last_change":"2024-04-25T11:38:25.674108+0000","last_active":"2024-04-25T11:38:26.762072+0000","last_peered":"2024-04-25T11:38:26.762072+0000","last_clean":"2024-04-25T11:38:26.762072+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:38:26.762072+0000","last_undegraded":"2024-04-25T11:38:26.762072+0000","last_fullsized":"2024-04-25T11:38:26.762072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761259+0000","last_change":"2024-04-25T11:38:25.672215+0000","last_active":"2024-04-25T11:38:26.761259+0000","last_peered":"2024-04-25T11:38:26.761259+0000","last_clean":"2024-04-25T11:38:26.761259+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:38:26.761259+0000","last_undegraded":"2024-04-25T11:38:26.761259+0000","last_fullsized":"2024-04-25T11:38:26.761259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674666+0000","last_change":"2024-04-25T11:38:25.679772+0000","last_active":"2024-04-25T11:38:26.674666+0000","last_peered":"2024-04-25T11:38:26.674666+0000","last_clean":"2024-04-25T11:38:26.674666+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:38:26.674666+0000","last_undegraded":"2024-04-25T11:38:26.674666+0000","last_fullsized":"2024-04-25T11:38:26.674666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673164+0000","last_change":"2024-04-25T11:38:25.665626+0000","last_active":"2024-04-25T11:38:26.673164+0000","last_peered":"2024-04-25T11:38:26.673164+0000","last_clean":"2024-04-25T11:38:26.673164+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:38:26.673164+0000","last_undegraded":"2024-04-25T11:38:26.673164+0000","last_fullsized":"2024-04-25T11:38:26.673164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761385+0000","last_change":"2024-04-25T11:38:25.672935+0000","last_active":"2024-04-25T11:38:26.761385+0000","last_peered":"2024-04-25T11:38:26.761385+0000","last_clean":"2024-04-25T11:38:26.761385+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:38:26.761385+0000","last_undegraded":"2024-04-25T11:38:26.761385+0000","last_fullsized":"2024-04-25T11:38:26.761385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761842+0000","last_change":"2024-04-25T11:38:25.679486+0000","last_active":"2024-04-25T11:38:26.761842+0000","last_peered":"2024-04-25T11:38:26.761842+0000","last_clean":"2024-04-25T11:38:26.761842+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:38:26.761842+0000","last_undegraded":"2024-04-25T11:38:26.761842+0000","last_fullsized":"2024-04-25T11:38:26.761842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762406+0000","last_change":"2024-04-25T11:38:25.679437+0000","last_active":"2024-04-25T11:38:26.762406+0000","last_peered":"2024-04-25T11:38:26.762406+0000","last_clean":"2024-04-25T11:38:26.762406+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:38:26.762406+0000","last_undegraded":"2024-04-25T11:38:26.762406+0000","last_fullsized":"2024-04-25T11:38:26.762406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673702+0000","last_change":"2024-04-25T11:38:25.670384+0000","last_active":"2024-04-25T11:38:26.673702+0000","last_peered":"2024-04-25T11:38:26.673702+0000","last_clean":"2024-04-25T11:38:26.673702+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:38:26.673702+0000","last_undegraded":"2024-04-25T11:38:26.673702+0000","last_fullsized":"2024-04-25T11:38:26.673702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674250+0000","last_change":"2024-04-25T11:38:25.680252+0000","last_active":"2024-04-25T11:38:26.674250+0000","last_peered":"2024-04-25T11:38:26.674250+0000","last_clean":"2024-04-25T11:38:26.674250+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:38:26.674250+0000","last_undegraded":"2024-04-25T11:38:26.674250+0000","last_fullsized":"2024-04-25T11:38:26.674250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762291+0000","last_change":"2024-04-25T11:38:25.669542+0000","last_active":"2024-04-25T11:38:26.762291+0000","last_peered":"2024-04-25T11:38:26.762291+0000","last_clean":"2024-04-25T11:38:26.762291+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:38:26.762291+0000","last_undegraded":"2024-04-25T11:38:26.762291+0000","last_fullsized":"2024-04-25T11:38:26.762291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:51.929330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761399+0000","last_change":"2024-04-25T11:38:24.666197+0000","last_active":"2024-04-25T11:38:26.761399+0000","last_peered":"2024-04-25T11:38:26.761399+0000","last_clean":"2024-04-25T11:38:26.761399+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:38:26.761399+0000","last_undegraded":"2024-04-25T11:38:26.761399+0000","last_fullsized":"2024-04-25T11:38:26.761399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.198972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761974+0000","last_change":"2024-04-25T11:38:25.682931+0000","last_active":"2024-04-25T11:38:26.761974+0000","last_peered":"2024-04-25T11:38:26.761974+0000","last_clean":"2024-04-25T11:38:26.761974+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:38:26.761974+0000","last_undegraded":"2024-04-25T11:38:26.761974+0000","last_fullsized":"2024-04-25T11:38:26.761974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674162+0000","last_change":"2024-04-25T11:38:24.660888+0000","last_active":"2024-04-25T11:38:26.674162+0000","last_peered":"2024-04-25T11:38:26.674162+0000","last_clean":"2024-04-25T11:38:26.674162+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:38:26.674162+0000","last_undegraded":"2024-04-25T11:38:26.674162+0000","last_fullsized":"2024-04-25T11:38:26.674162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761513+0000","last_change":"2024-04-25T11:38:24.657815+0000","last_active":"2024-04-25T11:38:26.761513+0000","last_peered":"2024-04-25T11:38:26.761513+0000","last_clean":"2024-04-25T11:38:26.761513+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:38:26.761513+0000","last_undegraded":"2024-04-25T11:38:26.761513+0000","last_fullsized":"2024-04-25T11:38:26.761513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761437+0000","last_change":"2024-04-25T11:38:25.680969+0000","last_active":"2024-04-25T11:38:26.761437+0000","last_peered":"2024-04-25T11:38:26.761437+0000","last_clean":"2024-04-25T11:38:26.761437+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:38:26.761437+0000","last_undegraded":"2024-04-25T11:38:26.761437+0000","last_fullsized":"2024-04-25T11:38:26.761437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762182+0000","last_change":"2024-04-25T11:38:24.668899+0000","last_active":"2024-04-25T11:38:26.762182+0000","last_peered":"2024-04-25T11:38:26.762182+0000","last_clean":"2024-04-25T11:38:26.762182+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:38:26.762182+0000","last_undegraded":"2024-04-25T11:38:26.762182+0000","last_fullsized":"2024-04-25T11:38:26.762182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672975+0000","last_change":"2024-04-25T11:38:25.666511+0000","last_active":"2024-04-25T11:38:26.672975+0000","last_peered":"2024-04-25T11:38:26.672975+0000","last_clean":"2024-04-25T11:38:26.672975+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:38:26.672975+0000","last_undegraded":"2024-04-25T11:38:26.672975+0000","last_fullsized":"2024-04-25T11:38:26.672975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674478+0000","last_change":"2024-04-25T11:38:24.666735+0000","last_active":"2024-04-25T11:38:26.674478+0000","last_peered":"2024-04-25T11:38:26.674478+0000","last_clean":"2024-04-25T11:38:26.674478+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:38:26.674478+0000","last_undegraded":"2024-04-25T11:38:26.674478+0000","last_fullsized":"2024-04-25T11:38:26.674478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763586+0000","last_change":"2024-04-25T11:38:25.680065+0000","last_active":"2024-04-25T11:38:26.763586+0000","last_peered":"2024-04-25T11:38:26.763586+0000","last_clean":"2024-04-25T11:38:26.763586+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:38:26.763586+0000","last_undegraded":"2024-04-25T11:38:26.763586+0000","last_fullsized":"2024-04-25T11:38:26.763586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763667+0000","last_change":"2024-04-25T11:38:24.671566+0000","last_active":"2024-04-25T11:38:26.763667+0000","last_peered":"2024-04-25T11:38:26.763667+0000","last_clean":"2024-04-25T11:38:26.763667+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:38:26.763667+0000","last_undegraded":"2024-04-25T11:38:26.763667+0000","last_fullsized":"2024-04-25T11:38:26.763667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761910+0000","last_change":"2024-04-25T11:38:25.678075+0000","last_active":"2024-04-25T11:38:26.761910+0000","last_peered":"2024-04-25T11:38:26.761910+0000","last_clean":"2024-04-25T11:38:26.761910+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:38:26.761910+0000","last_undegraded":"2024-04-25T11:38:26.761910+0000","last_fullsized":"2024-04-25T11:38:26.761910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673019+0000","last_change":"2024-04-25T11:38:24.667282+0000","last_active":"2024-04-25T11:38:26.673019+0000","last_peered":"2024-04-25T11:38:26.673019+0000","last_clean":"2024-04-25T11:38:26.673019+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:38:26.673019+0000","last_undegraded":"2024-04-25T11:38:26.673019+0000","last_fullsized":"2024-04-25T11:38:26.673019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762833+0000","last_change":"2024-04-25T11:38:25.680417+0000","last_active":"2024-04-25T11:38:26.762833+0000","last_peered":"2024-04-25T11:38:26.762833+0000","last_clean":"2024-04-25T11:38:26.762833+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:38:26.762833+0000","last_undegraded":"2024-04-25T11:38:26.762833+0000","last_fullsized":"2024-04-25T11:38:26.762833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674339+0000","last_change":"2024-04-25T11:38:24.662562+0000","last_active":"2024-04-25T11:38:26.674339+0000","last_peered":"2024-04-25T11:38:26.674339+0000","last_clean":"2024-04-25T11:38:26.674339+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:38:26.674339+0000","last_undegraded":"2024-04-25T11:38:26.674339+0000","last_fullsized":"2024-04-25T11:38:26.674339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762135+0000","last_change":"2024-04-25T11:38:25.677518+0000","last_active":"2024-04-25T11:38:26.762135+0000","last_peered":"2024-04-25T11:38:26.762135+0000","last_clean":"2024-04-25T11:38:26.762135+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:38:26.762135+0000","last_undegraded":"2024-04-25T11:38:26.762135+0000","last_fullsized":"2024-04-25T11:38:26.762135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761935+0000","last_change":"2024-04-25T11:38:24.658960+0000","last_active":"2024-04-25T11:38:26.761935+0000","last_peered":"2024-04-25T11:38:26.761935+0000","last_clean":"2024-04-25T11:38:26.761935+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:38:26.761935+0000","last_undegraded":"2024-04-25T11:38:26.761935+0000","last_fullsized":"2024-04-25T11:38:26.761935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761438+0000","last_change":"2024-04-25T11:38:25.681355+0000","last_active":"2024-04-25T11:38:26.761438+0000","last_peered":"2024-04-25T11:38:26.761438+0000","last_clean":"2024-04-25T11:38:26.761438+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:38:26.761438+0000","last_undegraded":"2024-04-25T11:38:26.761438+0000","last_fullsized":"2024-04-25T11:38:26.761438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762505+0000","last_change":"2024-04-25T11:38:24.666747+0000","last_active":"2024-04-25T11:38:26.762505+0000","last_peered":"2024-04-25T11:38:26.762505+0000","last_clean":"2024-04-25T11:38:26.762505+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:38:26.762505+0000","last_undegraded":"2024-04-25T11:38:26.762505+0000","last_fullsized":"2024-04-25T11:38:26.762505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762221+0000","last_change":"2024-04-25T11:38:25.684128+0000","last_active":"2024-04-25T11:38:26.762221+0000","last_peered":"2024-04-25T11:38:26.762221+0000","last_clean":"2024-04-25T11:38:26.762221+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:38:26.762221+0000","last_undegraded":"2024-04-25T11:38:26.762221+0000","last_fullsized":"2024-04-25T11:38:26.762221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674047+0000","last_change":"2024-04-25T11:38:24.661103+0000","last_active":"2024-04-25T11:38:26.674047+0000","last_peered":"2024-04-25T11:38:26.674047+0000","last_clean":"2024-04-25T11:38:26.674047+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:38:26.674047+0000","last_undegraded":"2024-04-25T11:38:26.674047+0000","last_fullsized":"2024-04-25T11:38:26.674047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674380+0000","last_change":"2024-04-25T11:38:25.680490+0000","last_active":"2024-04-25T11:38:26.674380+0000","last_peered":"2024-04-25T11:38:26.674380+0000","last_clean":"2024-04-25T11:38:26.674380+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:38:26.674380+0000","last_undegraded":"2024-04-25T11:38:26.674380+0000","last_fullsized":"2024-04-25T11:38:26.674380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673688+0000","last_change":"2024-04-25T11:38:24.665238+0000","last_active":"2024-04-25T11:38:26.673688+0000","last_peered":"2024-04-25T11:38:26.673688+0000","last_clean":"2024-04-25T11:38:26.673688+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:38:26.673688+0000","last_undegraded":"2024-04-25T11:38:26.673688+0000","last_fullsized":"2024-04-25T11:38:26.673688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761773+0000","last_change":"2024-04-25T11:38:25.682669+0000","last_active":"2024-04-25T11:38:26.761773+0000","last_peered":"2024-04-25T11:38:26.761773+0000","last_clean":"2024-04-25T11:38:26.761773+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:38:26.761773+0000","last_undegraded":"2024-04-25T11:38:26.761773+0000","last_fullsized":"2024-04-25T11:38:26.761773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762021+0000","last_change":"2024-04-25T11:38:24.658666+0000","last_active":"2024-04-25T11:38:26.762021+0000","last_peered":"2024-04-25T11:38:26.762021+0000","last_clean":"2024-04-25T11:38:26.762021+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:38:26.762021+0000","last_undegraded":"2024-04-25T11:38:26.762021+0000","last_fullsized":"2024-04-25T11:38:26.762021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:24.295273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762494+0000","last_change":"2024-04-25T11:38:25.680783+0000","last_active":"2024-04-25T11:38:26.762494+0000","last_peered":"2024-04-25T11:38:26.762494+0000","last_clean":"2024-04-25T11:38:26.762494+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:38:26.762494+0000","last_undegraded":"2024-04-25T11:38:26.762494+0000","last_fullsized":"2024-04-25T11:38:26.762494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673869+0000","last_change":"2024-04-25T11:38:24.656937+0000","last_active":"2024-04-25T11:38:26.673869+0000","last_peered":"2024-04-25T11:38:26.673869+0000","last_clean":"2024-04-25T11:38:26.673869+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:38:26.673869+0000","last_undegraded":"2024-04-25T11:38:26.673869+0000","last_fullsized":"2024-04-25T11:38:26.673869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763039+0000","last_change":"2024-04-25T11:38:25.681307+0000","last_active":"2024-04-25T11:38:26.763039+0000","last_peered":"2024-04-25T11:38:26.763039+0000","last_clean":"2024-04-25T11:38:26.763039+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:38:26.763039+0000","last_undegraded":"2024-04-25T11:38:26.763039+0000","last_fullsized":"2024-04-25T11:38:26.763039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:17.678183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673129+0000","last_change":"2024-04-25T11:38:24.663247+0000","last_active":"2024-04-25T11:38:26.673129+0000","last_peered":"2024-04-25T11:38:26.673129+0000","last_clean":"2024-04-25T11:38:26.673129+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:38:26.673129+0000","last_undegraded":"2024-04-25T11:38:26.673129+0000","last_fullsized":"2024-04-25T11:38:26.673129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763176+0000","last_change":"2024-04-25T11:38:25.676409+0000","last_active":"2024-04-25T11:38:26.763176+0000","last_peered":"2024-04-25T11:38:26.763176+0000","last_clean":"2024-04-25T11:38:26.763176+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.763176+0000","last_undegraded":"2024-04-25T11:38:26.763176+0000","last_fullsized":"2024-04-25T11:38:26.763176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673831+0000","last_change":"2024-04-25T11:38:24.666800+0000","last_active":"2024-04-25T11:38:26.673831+0000","last_peered":"2024-04-25T11:38:26.673831+0000","last_clean":"2024-04-25T11:38:26.673831+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:38:26.673831+0000","last_undegraded":"2024-04-25T11:38:26.673831+0000","last_fullsized":"2024-04-25T11:38:26.673831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762286+0000","last_change":"2024-04-25T11:38:25.678136+0000","last_active":"2024-04-25T11:38:26.762286+0000","last_peered":"2024-04-25T11:38:26.762286+0000","last_clean":"2024-04-25T11:38:26.762286+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:38:26.762286+0000","last_undegraded":"2024-04-25T11:38:26.762286+0000","last_fullsized":"2024-04-25T11:38:26.762286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761547+0000","last_change":"2024-04-25T11:38:20.865911+0000","last_active":"2024-04-25T11:38:26.761547+0000","last_peered":"2024-04-25T11:38:26.761547+0000","last_clean":"2024-04-25T11:38:26.761547+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:38:26.761547+0000","last_undegraded":"2024-04-25T11:38:26.761547+0000","last_fullsized":"2024-04-25T11:38:26.761547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_clean_scrub_stamp":"2024-04-25T11:38:19.606342+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:58.126842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762328+0000","last_change":"2024-04-25T11:38:24.671283+0000","last_active":"2024-04-25T11:38:26.762328+0000","last_peered":"2024-04-25T11:38:26.762328+0000","last_clean":"2024-04-25T11:38:26.762328+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:38:26.762328+0000","last_undegraded":"2024-04-25T11:38:26.762328+0000","last_fullsized":"2024-04-25T11:38:26.762328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761567+0000","last_change":"2024-04-25T11:38:25.675002+0000","last_active":"2024-04-25T11:38:26.761567+0000","last_peered":"2024-04-25T11:38:26.761567+0000","last_clean":"2024-04-25T11:38:26.761567+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:38:26.761567+0000","last_undegraded":"2024-04-25T11:38:26.761567+0000","last_fullsized":"2024-04-25T11:38:26.761567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761489+0000","last_change":"2024-04-25T11:38:24.671213+0000","last_active":"2024-04-25T11:38:26.761489+0000","last_peered":"2024-04-25T11:38:26.761489+0000","last_clean":"2024-04-25T11:38:26.761489+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:38:26.761489+0000","last_undegraded":"2024-04-25T11:38:26.761489+0000","last_fullsized":"2024-04-25T11:38:26.761489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761052+0000","last_change":"2024-04-25T11:38:25.673353+0000","last_active":"2024-04-25T11:38:26.761052+0000","last_peered":"2024-04-25T11:38:26.761052+0000","last_clean":"2024-04-25T11:38:26.761052+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:38:26.761052+0000","last_undegraded":"2024-04-25T11:38:26.761052+0000","last_fullsized":"2024-04-25T11:38:26.761052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:48.553601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673682+0000","last_change":"2024-04-25T11:38:24.650599+0000","last_active":"2024-04-25T11:38:26.673682+0000","last_peered":"2024-04-25T11:38:26.673682+0000","last_clean":"2024-04-25T11:38:26.673682+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:38:26.673682+0000","last_undegraded":"2024-04-25T11:38:26.673682+0000","last_fullsized":"2024-04-25T11:38:26.673682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672982+0000","last_change":"2024-04-25T11:38:25.668956+0000","last_active":"2024-04-25T11:38:26.672982+0000","last_peered":"2024-04-25T11:38:26.672982+0000","last_clean":"2024-04-25T11:38:26.672982+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:38:26.672982+0000","last_undegraded":"2024-04-25T11:38:26.672982+0000","last_fullsized":"2024-04-25T11:38:26.672982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762612+0000","last_change":"2024-04-25T11:38:24.665496+0000","last_active":"2024-04-25T11:38:26.762612+0000","last_peered":"2024-04-25T11:38:26.762612+0000","last_clean":"2024-04-25T11:38:26.762612+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:38:26.762612+0000","last_undegraded":"2024-04-25T11:38:26.762612+0000","last_fullsized":"2024-04-25T11:38:26.762612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762812+0000","last_change":"2024-04-25T11:38:25.681800+0000","last_active":"2024-04-25T11:38:26.762812+0000","last_peered":"2024-04-25T11:38:26.762812+0000","last_clean":"2024-04-25T11:38:26.762812+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:38:26.762812+0000","last_undegraded":"2024-04-25T11:38:26.762812+0000","last_fullsized":"2024-04-25T11:38:26.762812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:39.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673142+0000","last_change":"2024-04-25T11:38:24.659792+0000","last_active":"2024-04-25T11:38:26.673142+0000","last_peered":"2024-04-25T11:38:26.673142+0000","last_clean":"2024-04-25T11:38:26.673142+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:38:26.673142+0000","last_undegraded":"2024-04-25T11:38:26.673142+0000","last_fullsized":"2024-04-25T11:38:26.673142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763580+0000","last_change":"2024-04-25T11:38:25.673411+0000","last_active":"2024-04-25T11:38:26.763580+0000","last_peered":"2024-04-25T11:38:26.763580+0000","last_clean":"2024-04-25T11:38:26.763580+0000","last_became_active":"2024-04-25T11:38:25.673116+0000","last_became_peered":"2024-04-25T11:38:25.673116+0000","last_unstale":"2024-04-25T11:38:26.763580+0000","last_undegraded":"2024-04-25T11:38:26.763580+0000","last_fullsized":"2024-04-25T11:38:26.763580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762671+0000","last_change":"2024-04-25T11:38:24.671412+0000","last_active":"2024-04-25T11:38:26.762671+0000","last_peered":"2024-04-25T11:38:26.762671+0000","last_clean":"2024-04-25T11:38:26.762671+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:38:26.762671+0000","last_undegraded":"2024-04-25T11:38:26.762671+0000","last_fullsized":"2024-04-25T11:38:26.762671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:34.607841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673048+0000","last_change":"2024-04-25T11:38:25.668959+0000","last_active":"2024-04-25T11:38:26.673048+0000","last_peered":"2024-04-25T11:38:26.673048+0000","last_clean":"2024-04-25T11:38:26.673048+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:38:26.673048+0000","last_undegraded":"2024-04-25T11:38:26.673048+0000","last_fullsized":"2024-04-25T11:38:26.673048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673459+0000","last_change":"2024-04-25T11:38:24.664146+0000","last_active":"2024-04-25T11:38:26.673459+0000","last_peered":"2024-04-25T11:38:26.673459+0000","last_clean":"2024-04-25T11:38:26.673459+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:38:26.673459+0000","last_undegraded":"2024-04-25T11:38:26.673459+0000","last_fullsized":"2024-04-25T11:38:26.673459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761339+0000","last_change":"2024-04-25T11:38:25.680105+0000","last_active":"2024-04-25T11:38:26.761339+0000","last_peered":"2024-04-25T11:38:26.761339+0000","last_clean":"2024-04-25T11:38:26.761339+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:38:26.761339+0000","last_undegraded":"2024-04-25T11:38:26.761339+0000","last_fullsized":"2024-04-25T11:38:26.761339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674386+0000","last_change":"2024-04-25T11:38:24.668369+0000","last_active":"2024-04-25T11:38:26.674386+0000","last_peered":"2024-04-25T11:38:26.674386+0000","last_clean":"2024-04-25T11:38:26.674386+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:38:26.674386+0000","last_undegraded":"2024-04-25T11:38:26.674386+0000","last_fullsized":"2024-04-25T11:38:26.674386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761723+0000","last_change":"2024-04-25T11:38:25.676446+0000","last_active":"2024-04-25T11:38:26.761723+0000","last_peered":"2024-04-25T11:38:26.761723+0000","last_clean":"2024-04-25T11:38:26.761723+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.761723+0000","last_undegraded":"2024-04-25T11:38:26.761723+0000","last_fullsized":"2024-04-25T11:38:26.761723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761379+0000","last_change":"2024-04-25T11:38:24.671951+0000","last_active":"2024-04-25T11:38:26.761379+0000","last_peered":"2024-04-25T11:38:26.761379+0000","last_clean":"2024-04-25T11:38:26.761379+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:38:26.761379+0000","last_undegraded":"2024-04-25T11:38:26.761379+0000","last_fullsized":"2024-04-25T11:38:26.761379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761852+0000","last_change":"2024-04-25T11:38:25.669474+0000","last_active":"2024-04-25T11:38:26.761852+0000","last_peered":"2024-04-25T11:38:26.761852+0000","last_clean":"2024-04-25T11:38:26.761852+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:38:26.761852+0000","last_undegraded":"2024-04-25T11:38:26.761852+0000","last_fullsized":"2024-04-25T11:38:26.761852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:38.359208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673212+0000","last_change":"2024-04-25T11:38:24.668689+0000","last_active":"2024-04-25T11:38:26.673212+0000","last_peered":"2024-04-25T11:38:26.673212+0000","last_clean":"2024-04-25T11:38:26.673212+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:38:26.673212+0000","last_undegraded":"2024-04-25T11:38:26.673212+0000","last_fullsized":"2024-04-25T11:38:26.673212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761421+0000","last_change":"2024-04-25T11:38:25.668088+0000","last_active":"2024-04-25T11:38:26.761421+0000","last_peered":"2024-04-25T11:38:26.761421+0000","last_clean":"2024-04-25T11:38:26.761421+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:38:26.761421+0000","last_undegraded":"2024-04-25T11:38:26.761421+0000","last_fullsized":"2024-04-25T11:38:26.761421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673373+0000","last_change":"2024-04-25T11:38:24.662913+0000","last_active":"2024-04-25T11:38:26.673373+0000","last_peered":"2024-04-25T11:38:26.673373+0000","last_clean":"2024-04-25T11:38:26.673373+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:38:26.673373+0000","last_undegraded":"2024-04-25T11:38:26.673373+0000","last_fullsized":"2024-04-25T11:38:26.673373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761308+0000","last_change":"2024-04-25T11:38:25.680244+0000","last_active":"2024-04-25T11:38:26.761308+0000","last_peered":"2024-04-25T11:38:26.761308+0000","last_clean":"2024-04-25T11:38:26.761308+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:38:26.761308+0000","last_undegraded":"2024-04-25T11:38:26.761308+0000","last_fullsized":"2024-04-25T11:38:26.761308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762404+0000","last_change":"2024-04-25T11:38:24.669196+0000","last_active":"2024-04-25T11:38:26.762404+0000","last_peered":"2024-04-25T11:38:26.762404+0000","last_clean":"2024-04-25T11:38:26.762404+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:38:26.762404+0000","last_undegraded":"2024-04-25T11:38:26.762404+0000","last_fullsized":"2024-04-25T11:38:26.762404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.791358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674198+0000","last_change":"2024-04-25T11:38:25.668400+0000","last_active":"2024-04-25T11:38:26.674198+0000","last_peered":"2024-04-25T11:38:26.674198+0000","last_clean":"2024-04-25T11:38:26.674198+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:38:26.674198+0000","last_undegraded":"2024-04-25T11:38:26.674198+0000","last_fullsized":"2024-04-25T11:38:26.674198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763520+0000","last_change":"2024-04-25T11:38:25.683663+0000","last_active":"2024-04-25T11:38:26.763520+0000","last_peered":"2024-04-25T11:38:26.763520+0000","last_clean":"2024-04-25T11:38:26.763520+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:38:26.763520+0000","last_undegraded":"2024-04-25T11:38:26.763520+0000","last_fullsized":"2024-04-25T11:38:26.763520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761944+0000","last_change":"2024-04-25T11:38:24.670671+0000","last_active":"2024-04-25T11:38:26.761944+0000","last_peered":"2024-04-25T11:38:26.761944+0000","last_clean":"2024-04-25T11:38:26.761944+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:38:26.761944+0000","last_undegraded":"2024-04-25T11:38:26.761944+0000","last_fullsized":"2024-04-25T11:38:26.761944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763017+0000","last_change":"2024-04-25T11:38:25.681439+0000","last_active":"2024-04-25T11:38:26.763017+0000","last_peered":"2024-04-25T11:38:26.763017+0000","last_clean":"2024-04-25T11:38:26.763017+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:38:26.763017+0000","last_undegraded":"2024-04-25T11:38:26.763017+0000","last_fullsized":"2024-04-25T11:38:26.763017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762567+0000","last_change":"2024-04-25T11:38:24.669485+0000","last_active":"2024-04-25T11:38:26.762567+0000","last_peered":"2024-04-25T11:38:26.762567+0000","last_clean":"2024-04-25T11:38:26.762567+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:38:26.762567+0000","last_undegraded":"2024-04-25T11:38:26.762567+0000","last_fullsized":"2024-04-25T11:38:26.762567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761766+0000","last_change":"2024-04-25T11:38:25.684415+0000","last_active":"2024-04-25T11:38:26.761766+0000","last_peered":"2024-04-25T11:38:26.761766+0000","last_clean":"2024-04-25T11:38:26.761766+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:38:26.761766+0000","last_undegraded":"2024-04-25T11:38:26.761766+0000","last_fullsized":"2024-04-25T11:38:26.761766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761477+0000","last_change":"2024-04-25T11:38:24.658741+0000","last_active":"2024-04-25T11:38:26.761477+0000","last_peered":"2024-04-25T11:38:26.761477+0000","last_clean":"2024-04-25T11:38:26.761477+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:38:26.761477+0000","last_undegraded":"2024-04-25T11:38:26.761477+0000","last_fullsized":"2024-04-25T11:38:26.761477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761168+0000","last_change":"2024-04-25T11:38:25.667505+0000","last_active":"2024-04-25T11:38:26.761168+0000","last_peered":"2024-04-25T11:38:26.761168+0000","last_clean":"2024-04-25T11:38:26.761168+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:38:26.761168+0000","last_undegraded":"2024-04-25T11:38:26.761168+0000","last_fullsized":"2024-04-25T11:38:26.761168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761936+0000","last_change":"2024-04-25T11:38:24.669215+0000","last_active":"2024-04-25T11:38:26.761936+0000","last_peered":"2024-04-25T11:38:26.761936+0000","last_clean":"2024-04-25T11:38:26.761936+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:38:26.761936+0000","last_undegraded":"2024-04-25T11:38:26.761936+0000","last_fullsized":"2024-04-25T11:38:26.761936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":661302,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":661302,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:38:31.043 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:38:31.044 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T11:38:31.044 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T11:38:31.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T11:38:31.044 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T11:38:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.192+0000 7fa3748fc640 1 Processor -- start 2024-04-25T11:38:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.192+0000 7fa3748fc640 1 -- start start 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.193+0000 7fa3748fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa370155080 0x7fa3700cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.193+0000 7fa3748fc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3700cb460 con 0x7fa3700c7840 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.193+0000 7fa3748fc640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3700cb740 con 0x7fa37015ed40 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.193+0000 7fa3748fc640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa37015af50 con 0x7fa370155080 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.193+0000 7fa36ed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa3700c7840 0x7fa3700d03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43900/0 (socket says 172.21.15.40:43900) 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.193+0000 7fa36ed9d640 1 -- 172.21.15.40:0/2646421661 learned_addr learned my addr 172.21.15.40:0/2646421661 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.193+0000 7fa36e59c640 1 --2- 172.21.15.40:0/2646421661 >> v2:172.21.15.62:3300/0 conn(0x7fa370155080 0x7fa3700cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.193+0000 7fa36d59a640 1 -- 172.21.15.40:0/2646421661 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4090463950 0 0) 0x7fa3700cb460 con 0x7fa3700c7840 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.193+0000 7fa36d59a640 1 -- 172.21.15.40:0/2646421661 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa358003520 con 0x7fa3700c7840 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.194+0000 7fa36d59a640 1 -- 172.21.15.40:0/2646421661 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2104020109 0 0) 0x7fa358003520 con 0x7fa3700c7840 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.194+0000 7fa36e59c640 1 -- 172.21.15.40:0/2646421661 >> v1:172.21.15.62:6789/0 conn(0x7fa37015ed40 legacy=0x7fa37015b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.194+0000 7fa36e59c640 1 -- 172.21.15.40:0/2646421661 >> v1:172.21.15.40:6789/0 conn(0x7fa3700c7840 legacy=0x7fa3700d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.194+0000 7fa36e59c640 1 -- 172.21.15.40:0/2646421661 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37015b230 con 0x7fa370155080 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.194+0000 7fa36e59c640 1 --2- 172.21.15.40:0/2646421661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa370155080 0x7fa3700cc720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa3600097a0 tx=0x7fa36002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d21dfd741561078c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa36d59a640 1 -- 172.21.15.40:0/2646421661 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360037070 con 0x7fa370155080 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa36d59a640 1 -- 172.21.15.40:0/2646421661 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36002abc0 con 0x7fa370155080 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa36d59a640 1 -- 172.21.15.40:0/2646421661 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368001a10 con 0x7fa370155080 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa3748fc640 1 -- 172.21.15.40:0/2646421661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa370155080 msgr2=0x7fa3700cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa3748fc640 1 --2- 172.21.15.40:0/2646421661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa370155080 0x7fa3700cc720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa3600097a0 tx=0x7fa36002a130 comp rx=0 tx=0).stop 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa3748fc640 1 -- 172.21.15.40:0/2646421661 shutdown_connections 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa3748fc640 1 --2- 172.21.15.40:0/2646421661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa370155080 0x7fa3700cc720 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa3748fc640 1 -- 172.21.15.40:0/2646421661 >> 172.21.15.40:0/2646421661 conn(0x7fa37006c680 msgr2=0x7fa37006e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa3748fc640 1 -- 172.21.15.40:0/2646421661 shutdown_connections 2024-04-25T11:38:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.195+0000 7fa3748fc640 1 -- 172.21.15.40:0/2646421661 wait complete. 2024-04-25T11:38:31.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.196+0000 7fa3748fc640 1 Processor -- start 2024-04-25T11:38:31.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa3748fc640 1 -- start start 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa3748fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3700c7840 0x7fa370150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa3748fc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa370155080 0x7fa370150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa3748fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37015ed40 0x7fa3701f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa3748fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3700d0ae0 con 0x7fa3700c7840 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa3748fc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa37015f210 con 0x7fa37015ed40 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa3748fc640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa3700d0c60 con 0x7fa370155080 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa36e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3700c7840 0x7fa370150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa36e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3700c7840 0x7fa370150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57504/0 (socket says 172.21.15.40:57504) 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.197+0000 7fa36e59c640 1 -- 172.21.15.40:0/2777039399 learned_addr learned my addr 172.21.15.40:0/2777039399 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa36e59c640 1 -- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa370155080 msgr2=0x7fa370150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa36dd9b640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa370155080 0x7fa370150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa36ed9d640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37015ed40 0x7fa3701f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa36e59c640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa370155080 0x7fa370150c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa36e59c640 1 -- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37015ed40 msgr2=0x7fa3701f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa36e59c640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37015ed40 0x7fa3701f1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa36e59c640 1 -- 172.21.15.40:0/2777039399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa370153440 con 0x7fa3700c7840 2024-04-25T11:38:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa36e59c640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3700c7840 0x7fa370150730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa360035040 tx=0x7fa3600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa35effd640 1 -- 172.21.15.40:0/2777039399 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360037040 con 0x7fa3700c7840 2024-04-25T11:38:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa35effd640 1 -- 172.21.15.40:0/2777039399 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa360004230 con 0x7fa3700c7840 2024-04-25T11:38:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa35effd640 1 -- 172.21.15.40:0/2777039399 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360003610 con 0x7fa3700c7840 2024-04-25T11:38:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3701566c0 con 0x7fa3700c7840 2024-04-25T11:38:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.198+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa370156c80 con 0x7fa3700c7840 2024-04-25T11:38:31.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.199+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa370068bb0 con 0x7fa3700c7840 2024-04-25T11:38:31.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.200+0000 7fa35effd640 1 -- 172.21.15.40:0/2777039399 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3600037b0 con 0x7fa3700c7840 2024-04-25T11:38:31.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.200+0000 7fa35effd640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa35006c8f0 0x7fa35006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.200+0000 7fa35effd640 1 -- 172.21.15.40:0/2777039399 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3600acb00 con 0x7fa3700c7840 2024-04-25T11:38:31.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.200+0000 7fa36dd9b640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa35006c8f0 0x7fa35006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.201+0000 7fa36dd9b640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa35006c8f0 0x7fa35006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa370153b40 tx=0x7fa364009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.204+0000 7fa35effd640 1 -- 172.21.15.40:0/2777039399 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa36007ae90 con 0x7fa3700c7840 2024-04-25T11:38:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.438+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa3700d04d0 con 0x7fa3700c7840 2024-04-25T11:38:31.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.441+0000 7fa35effd640 1 -- 172.21.15.40:0/2777039399 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174136 (secure 0 0 0) 0x7fa36007a790 con 0x7fa3700c7840 2024-04-25T11:38:31.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:31.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa35006c8f0 msgr2=0x7fa35006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa35006c8f0 0x7fa35006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa370153b40 tx=0x7fa364009040 comp rx=0 tx=0).stop 2024-04-25T11:38:31.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3700c7840 msgr2=0x7fa370150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3700c7840 0x7fa370150730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa360035040 tx=0x7fa3600046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 shutdown_connections 2024-04-25T11:38:31.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa35006c8f0 0x7fa35006edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa37015ed40 0x7fa3701f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa370155080 0x7fa370150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 --2- 172.21.15.40:0/2777039399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3700c7840 0x7fa370150730 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 >> 172.21.15.40:0/2777039399 conn(0x7fa37006c680 msgr2=0x7fa370151e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:31.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 shutdown_connections 2024-04-25T11:38:31.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.444+0000 7fa3748fc640 1 -- 172.21.15.40:0/2777039399 wait complete. 2024-04-25T11:38:31.443 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1406383768},{"type":"v1","addr":"172.21.15.62:6833","nonce":1406383768}]},"active_addr":"172.21.15.62:6833/1406383768","active_change":"2024-04-25T11:38:12.944079+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4117,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":3870364761}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1923500293}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":2179537807}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":823515693}]}]} 2024-04-25T11:38:31.446 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T11:38:31.446 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T11:38:31.446 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.558+0000 7fed3e722640 1 Processor -- start 2024-04-25T11:38:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.558+0000 7fed3e722640 1 -- start start 2024-04-25T11:38:31.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed3e722640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed380c9210 0x7fed38159590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed3e722640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed380c9f40 con 0x7fed38152b00 2024-04-25T11:38:31.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed3e722640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed38158af0 con 0x7fed38152ee0 2024-04-25T11:38:31.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed3e722640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fed380c4910 con 0x7fed380c9210 2024-04-25T11:38:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed37fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fed38152b00 0x7fed380c3480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43906/0 (socket says 172.21.15.40:43906) 2024-04-25T11:38:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed3ccbf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed380c9210 0x7fed38159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed37fff640 1 -- 172.21.15.40:0/1251706715 learned_addr learned my addr 172.21.15.40:0/1251706715 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed3ccbf640 1 -- 172.21.15.40:0/1251706715 >> v1:172.21.15.62:6789/0 conn(0x7fed38152ee0 legacy=0x7fed380ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed3ccbf640 1 -- 172.21.15.40:0/1251706715 >> v1:172.21.15.40:6789/0 conn(0x7fed38152b00 legacy=0x7fed380c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed36ffd640 1 -- 172.21.15.40:0/1251706715 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644443598 0 0) 0x7fed380c9f40 con 0x7fed38152b00 2024-04-25T11:38:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.559+0000 7fed3ccbf640 1 -- 172.21.15.40:0/1251706715 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed380ca1e0 con 0x7fed380c9210 2024-04-25T11:38:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.560+0000 7fed3ccbf640 1 --2- 172.21.15.40:0/1251706715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed380c9210 0x7fed38159590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d90 tx=0x7fed3002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea736728d3307c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.560+0000 7fed36ffd640 1 -- 172.21.15.40:0/1251706715 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30007740 con 0x7fed380c9210 2024-04-25T11:38:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.560+0000 7fed36ffd640 1 -- 172.21.15.40:0/1251706715 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed300078e0 con 0x7fed380c9210 2024-04-25T11:38:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.560+0000 7fed36ffd640 1 -- 172.21.15.40:0/1251706715 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30007cb0 con 0x7fed380c9210 2024-04-25T11:38:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.561+0000 7fed3e722640 1 -- 172.21.15.40:0/1251706715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed380c9210 msgr2=0x7fed38159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.561+0000 7fed3e722640 1 --2- 172.21.15.40:0/1251706715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed380c9210 0x7fed38159590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d90 tx=0x7fed3002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.561+0000 7fed3e722640 1 -- 172.21.15.40:0/1251706715 shutdown_connections 2024-04-25T11:38:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.561+0000 7fed3e722640 1 --2- 172.21.15.40:0/1251706715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed380c9210 0x7fed38159590 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.561+0000 7fed3e722640 1 -- 172.21.15.40:0/1251706715 >> 172.21.15.40:0/1251706715 conn(0x7fed3806f9c0 msgr2=0x7fed380676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.561+0000 7fed3e722640 1 -- 172.21.15.40:0/1251706715 shutdown_connections 2024-04-25T11:38:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.561+0000 7fed3e722640 1 -- 172.21.15.40:0/1251706715 wait complete. 2024-04-25T11:38:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.562+0000 7fed3e722640 1 Processor -- start 2024-04-25T11:38:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.562+0000 7fed3e722640 1 -- start start 2024-04-25T11:38:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3e722640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed380c9210 0x7fed3806da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3e722640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed38152b00 0x7fed381f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3e722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed3806e610 0x7fed381f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3e722640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed380c8e60 con 0x7fed3806e610 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3e722640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed38159100 con 0x7fed380c9210 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3e722640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fed380ce3f0 con 0x7fed38152b00 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3ccbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed3806e610 0x7fed381f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3ccbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed3806e610 0x7fed381f0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57514/0 (socket says 172.21.15.40:57514) 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3ccbf640 1 -- 172.21.15.40:0/2452576893 learned_addr learned my addr 172.21.15.40:0/2452576893 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed37fff640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed380c9210 0x7fed3806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed377fe640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed38152b00 0x7fed381f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3ccbf640 1 -- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed38152b00 msgr2=0x7fed381f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3ccbf640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed38152b00 0x7fed381f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3ccbf640 1 -- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed380c9210 msgr2=0x7fed3806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3ccbf640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed380c9210 0x7fed3806da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3ccbf640 1 -- 172.21.15.40:0/2452576893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed380c83b0 con 0x7fed3806e610 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed377fe640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed38152b00 0x7fed381f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3ccbf640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed3806e610 0x7fed381f0ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed30009eb0 tx=0x7fed30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed34ff9640 1 -- 172.21.15.40:0/2452576893 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30040070 con 0x7fed3806e610 2024-04-25T11:38:31.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed34ff9640 1 -- 172.21.15.40:0/2452576893 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed30031c40 con 0x7fed3806e610 2024-04-25T11:38:31.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed380c6990 con 0x7fed3806e610 2024-04-25T11:38:31.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed34ff9640 1 -- 172.21.15.40:0/2452576893 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30003990 con 0x7fed3806e610 2024-04-25T11:38:31.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.563+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed380c4440 con 0x7fed3806e610 2024-04-25T11:38:31.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.564+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed38069200 con 0x7fed3806e610 2024-04-25T11:38:31.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.565+0000 7fed34ff9640 1 -- 172.21.15.40:0/2452576893 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed30039040 con 0x7fed3806e610 2024-04-25T11:38:31.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.565+0000 7fed34ff9640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed1806c8f0 0x7fed1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.565+0000 7fed34ff9640 1 -- 172.21.15.40:0/2452576893 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fed300acd90 con 0x7fed3806e610 2024-04-25T11:38:31.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.565+0000 7fed37fff640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed1806c8f0 0x7fed1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.566+0000 7fed37fff640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed1806c8f0 0x7fed1806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed28007f70 tx=0x7fed28007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.569+0000 7fed34ff9640 1 -- 172.21.15.40:0/2452576893 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed3007ac00 con 0x7fed3806e610 2024-04-25T11:38:31.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.756+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed380cd2a0 con 0x7fed3806e610 2024-04-25T11:38:31.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.758+0000 7fed34ff9640 1 -- 172.21.15.40:0/2452576893 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fed3007ef00 con 0x7fed3806e610 2024-04-25T11:38:31.746 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:31.747 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:26.664101+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed1806c8f0 msgr2=0x7fed1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed1806c8f0 0x7fed1806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed28007f70 tx=0x7fed28007f00 comp rx=0 tx=0).stop 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed3806e610 msgr2=0x7fed381f0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed3806e610 0x7fed381f0ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed30009eb0 tx=0x7fed30004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 shutdown_connections 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed1806c8f0 0x7fed1806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed3806e610 0x7fed381f0ae0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed38152b00 0x7fed381f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 --2- 172.21.15.40:0/2452576893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed380c9210 0x7fed3806da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.759+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 >> 172.21.15.40:0/2452576893 conn(0x7fed3806f9c0 msgr2=0x7fed380ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.760+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 shutdown_connections 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.760+0000 7fed3e722640 1 -- 172.21.15.40:0/2452576893 wait complete. 2024-04-25T11:38:31.759 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T11:38:31.759 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:31.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.871+0000 7fa31788a640 1 Processor -- start 2024-04-25T11:38:31.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.871+0000 7fa31788a640 1 -- start start 2024-04-25T11:38:31.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa31788a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa310154f50 0x7fa3100c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa31788a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3100b6210 0x7fa3100cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa31788a640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa31015f060 con 0x7fa3100cb7e0 2024-04-25T11:38:31.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa31788a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa31015eb20 con 0x7fa310154f50 2024-04-25T11:38:31.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa31788a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa3100d1040 con 0x7fa3100b6210 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa314e25640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3100b6210 0x7fa3100cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa315626640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa310154f50 0x7fa3100c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa315e27640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa3100cb7e0 0x7fa3100c8690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:54252/0 (socket says 172.21.15.40:54252) 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa315626640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa310154f50 0x7fa3100c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51452/0 (socket says 172.21.15.40:51452) 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa314e25640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3100b6210 0x7fa3100cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40662/0 (socket says 172.21.15.40:40662) 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.872+0000 7fa315e27640 1 -- 172.21.15.40:0/1811152224 learned_addr learned my addr 172.21.15.40:0/1811152224 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.873+0000 7fa315626640 1 -- 172.21.15.40:0/1811152224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3100b6210 msgr2=0x7fa3100cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.873+0000 7fa315626640 1 --2- 172.21.15.40:0/1811152224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3100b6210 0x7fa3100cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.873+0000 7fa315626640 1 -- 172.21.15.40:0/1811152224 >> v1:172.21.15.62:6789/0 conn(0x7fa3100cb7e0 legacy=0x7fa3100c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.873+0000 7fa315626640 1 -- 172.21.15.40:0/1811152224 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3100cc150 con 0x7fa310154f50 2024-04-25T11:38:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.873+0000 7fa303fff640 1 -- 172.21.15.40:0/1811152224 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699555273 0 0) 0x7fa31015f060 con 0x7fa3100cb7e0 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.873+0000 7fa315626640 1 --2- 172.21.15.40:0/1811152224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310154f50 0x7fa3100c7a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3040097a0 tx=0x7fa30402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ba7184a7fe50e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.873+0000 7fa303fff640 1 -- 172.21.15.40:0/1811152224 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304037070 con 0x7fa310154f50 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.873+0000 7fa303fff640 1 -- 172.21.15.40:0/1811152224 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30402ae50 con 0x7fa310154f50 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.873+0000 7fa303fff640 1 -- 172.21.15.40:0/1811152224 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304032660 con 0x7fa310154f50 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.874+0000 7fa31788a640 1 -- 172.21.15.40:0/1811152224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310154f50 msgr2=0x7fa3100c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.874+0000 7fa31788a640 1 --2- 172.21.15.40:0/1811152224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310154f50 0x7fa3100c7a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3040097a0 tx=0x7fa30402a100 comp rx=0 tx=0).stop 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.874+0000 7fa31788a640 1 -- 172.21.15.40:0/1811152224 shutdown_connections 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.874+0000 7fa31788a640 1 --2- 172.21.15.40:0/1811152224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3100b6210 0x7fa3100cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.874+0000 7fa31788a640 1 --2- 172.21.15.40:0/1811152224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310154f50 0x7fa3100c7a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.874+0000 7fa31788a640 1 -- 172.21.15.40:0/1811152224 >> 172.21.15.40:0/1811152224 conn(0x7fa3100d0440 msgr2=0x7fa3100d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.874+0000 7fa31788a640 1 -- 172.21.15.40:0/1811152224 shutdown_connections 2024-04-25T11:38:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.874+0000 7fa31788a640 1 -- 172.21.15.40:0/1811152224 wait complete. 2024-04-25T11:38:31.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.875+0000 7fa31788a640 1 Processor -- start 2024-04-25T11:38:31.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.875+0000 7fa31788a640 1 -- start start 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa31788a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3100cb7e0 0x7fa31015c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa31788a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa310154f50 0x7fa31015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa31788a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3101f41a0 0x7fa3101f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa31788a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3100cc3f0 con 0x7fa310154f50 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa31788a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa31015ed80 con 0x7fa3100cb7e0 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa31788a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa3100c8170 con 0x7fa3101f41a0 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa314e25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa310154f50 0x7fa31015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa314e25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa310154f50 0x7fa31015cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57528/0 (socket says 172.21.15.40:57528) 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa314e25640 1 -- 172.21.15.40:0/389331230 learned_addr learned my addr 172.21.15.40:0/389331230 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa315626640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3100cb7e0 0x7fa31015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa315e27640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3101f41a0 0x7fa3101f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa314e25640 1 -- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3101f41a0 msgr2=0x7fa3101f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa314e25640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3101f41a0 0x7fa3101f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa314e25640 1 -- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3100cb7e0 msgr2=0x7fa31015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa314e25640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3100cb7e0 0x7fa31015c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.876+0000 7fa314e25640 1 -- 172.21.15.40:0/389331230 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3100cf6d0 con 0x7fa310154f50 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.877+0000 7fa315e27640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3101f41a0 0x7fa3101f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.877+0000 7fa314e25640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa310154f50 0x7fa31015cd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e870 tx=0x7fa30c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.877+0000 7fa301ffb640 1 -- 172.21.15.40:0/389331230 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c017070 con 0x7fa310154f50 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.877+0000 7fa301ffb640 1 -- 172.21.15.40:0/389331230 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30c00c940 con 0x7fa310154f50 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.877+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3100cfeb0 con 0x7fa310154f50 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.877+0000 7fa301ffb640 1 -- 172.21.15.40:0/389331230 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00cd30 con 0x7fa310154f50 2024-04-25T11:38:31.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.877+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31015bd30 con 0x7fa310154f50 2024-04-25T11:38:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.878+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa310068bb0 con 0x7fa310154f50 2024-04-25T11:38:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.879+0000 7fa301ffb640 1 -- 172.21.15.40:0/389331230 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa30c01a070 con 0x7fa310154f50 2024-04-25T11:38:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.879+0000 7fa301ffb640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa2f406c8f0 0x7fa2f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.879+0000 7fa301ffb640 1 -- 172.21.15.40:0/389331230 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa30c08b990 con 0x7fa310154f50 2024-04-25T11:38:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.879+0000 7fa315626640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa2f406c8f0 0x7fa2f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.880+0000 7fa315626640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa2f406c8f0 0x7fa2f406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa304034000 tx=0x7fa30402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:31.883+0000 7fa301ffb640 1 -- 172.21.15.40:0/389331230 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa30c059e50 con 0x7fa310154f50 2024-04-25T11:38:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.070+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3100cbe00 con 0x7fa310154f50 2024-04-25T11:38:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.071+0000 7fa301ffb640 1 -- 172.21.15.40:0/389331230 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fa30c059750 con 0x7fa310154f50 2024-04-25T11:38:32.060 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:32.060 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:26.664101+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa2f406c8f0 msgr2=0x7fa2f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa2f406c8f0 0x7fa2f406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa304034000 tx=0x7fa30402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa310154f50 msgr2=0x7fa31015cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa310154f50 0x7fa31015cd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e870 tx=0x7fa30c00ec70 comp rx=0 tx=0).stop 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 shutdown_connections 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa2f406c8f0 0x7fa2f406edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3101f41a0 0x7fa3101f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa310154f50 0x7fa31015cd50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 --2- 172.21.15.40:0/389331230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3100cb7e0 0x7fa31015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 >> 172.21.15.40:0/389331230 conn(0x7fa3100d0440 msgr2=0x7fa3100c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 shutdown_connections 2024-04-25T11:38:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.073+0000 7fa31788a640 1 -- 172.21.15.40:0/389331230 wait complete. 2024-04-25T11:38:32.073 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T11:38:32.073 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T11:38:32.073 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T11:38:32.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T11:38:32.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T11:38:32.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T11:38:32.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T11:38:32.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T11:38:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.259+0000 7fa1220e1640 1 Processor -- start 2024-04-25T11:38:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.259+0000 7f0d7323c640 1 Processor -- start 2024-04-25T11:38:32.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.260+0000 7f0d7323c640 1 -- start start 2024-04-25T11:38:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.259+0000 7fa1220e1640 1 -- start start 2024-04-25T11:38:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7f0d7323c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0d6c06d300 0x7f0d6c15fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7fa1220e1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa11c1595d0 0x7fa11c1605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7fa1220e1640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c15f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7fa1220e1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa11c15fc80 con 0x7fa11c15f6a0 2024-04-25T11:38:32.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7fa1220e1640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa11c162f70 con 0x7fa11c1595d0 2024-04-25T11:38:32.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7fa1220e1640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c1626d0 con 0x7fa11c0cb120 2024-04-25T11:38:32.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7f0d7323c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d6c06d6e0 0x7f0d6c06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7f0d7323c640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d6c159840 con 0x7f0d6c0c3c60 2024-04-25T11:38:32.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.260+0000 7f5f0635d640 1 Processor -- start 2024-04-25T11:38:32.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7f5f0635d640 1 -- start start 2024-04-25T11:38:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7f0d7323c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0d6c06cfc0 con 0x7f0d6c06d6e0 2024-04-25T11:38:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7f0d7323c640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f0d6c06f5f0 con 0x7f0d6c06d300 2024-04-25T11:38:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7fa11affd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c15f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7fa11affd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c15f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40682/0 (socket says 172.21.15.40:40682) 2024-04-25T11:38:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7fa11affd640 1 -- 172.21.15.40:0/2086588747 learned_addr learned my addr 172.21.15.40:0/2086588747 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7fa11b7fe640 1 --2- 172.21.15.40:0/2086588747 >> v2:172.21.15.62:3300/0 conn(0x7fa11c1595d0 0x7fa11c1605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7f0d6bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d6c06d6e0 0x7f0d6c06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f5f0635d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ef80d5200 0x7f5ef804b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f5f0635d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ef804b090 0x7f5ef80db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f5f0635d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef80dbd70 con 0x7f5ef8005ee0 2024-04-25T11:38:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f5f0635d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef8045ed0 con 0x7f5ef80d5200 2024-04-25T11:38:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f5f0635d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef8046f30 con 0x7f5ef804b090 2024-04-25T11:38:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f0d6bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d6c06d6e0 0x7f0d6c06ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57532/0 (socket says 172.21.15.40:57532) 2024-04-25T11:38:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f0d6bfff640 1 -- 172.21.15.40:0/837088096 learned_addr learned my addr 172.21.15.40:0/837088096 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f0d6bfff640 1 -- 172.21.15.40:0/837088096 >> v2:172.21.15.62:3300/0 conn(0x7f0d6c06d300 msgr2=0x7f0d6c15fb30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:38:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f0d6bfff640 1 --2- 172.21.15.40:0/837088096 >> v2:172.21.15.62:3300/0 conn(0x7f0d6c06d300 0x7f0d6c15fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f0d6bfff640 1 -- 172.21.15.40:0/837088096 >> v1:172.21.15.62:6789/0 conn(0x7f0d6c0c3c60 legacy=0x7f0d6c0c4110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:38:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7f0d6bfff640 1 -- 172.21.15.40:0/837088096 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6c06f840 con 0x7f0d6c06d6e0 2024-04-25T11:38:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7fce68d6f640 1 Processor -- start 2024-04-25T11:38:32.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.261+0000 7fce68d6f640 1 -- start start 2024-04-25T11:38:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7fa11affd640 1 -- 172.21.15.40:0/2086588747 >> v2:172.21.15.62:3300/0 conn(0x7fa11c1595d0 msgr2=0x7fa11c1605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7fa11affd640 1 --2- 172.21.15.40:0/2086588747 >> v2:172.21.15.62:3300/0 conn(0x7fa11c1595d0 0x7fa11c1605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7fa11affd640 1 -- 172.21.15.40:0/2086588747 >> v1:172.21.15.62:6789/0 conn(0x7fa11c15f6a0 legacy=0x7fa11c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.262+0000 7fa11affd640 1 -- 172.21.15.40:0/2086588747 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c15ff60 con 0x7fa11c0cb120 2024-04-25T11:38:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7f5f0535b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ef80d5200 0x7f5ef804b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7f5f0535b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ef80d5200 0x7f5ef804b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57538/0 (socket says 172.21.15.40:57538) 2024-04-25T11:38:32.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7f5f0535b640 1 -- 172.21.15.40:0/3648975492 learned_addr learned my addr 172.21.15.40:0/3648975492 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce68d6f640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c045710 0x7fce5c045b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce68d6f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce5c0de3a0 0x7fce5c0de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce68d6f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce5c006ed0 con 0x7fce5c0d5080 2024-04-25T11:38:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce68d6f640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fce5c007ae0 con 0x7fce5c0de3a0 2024-04-25T11:38:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce68d6f640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c0071b0 con 0x7fce5c045710 2024-04-25T11:38:32.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7f5f04b5a640 1 --2- 172.21.15.40:0/3648975492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ef804b090 0x7f5ef80db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce62d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce5c0de3a0 0x7fce5c0de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7f5f0535b640 1 -- 172.21.15.40:0/3648975492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ef804b090 msgr2=0x7f5ef80db830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7f5f0535b640 1 --2- 172.21.15.40:0/3648975492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ef804b090 0x7f5ef80db830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7f5f0535b640 1 -- 172.21.15.40:0/3648975492 >> v1:172.21.15.40:6789/0 conn(0x7f5ef8005ee0 legacy=0x7f5ef8045330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7f5f0535b640 1 -- 172.21.15.40:0/3648975492 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef8046170 con 0x7f5ef80d5200 2024-04-25T11:38:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce62d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce5c0de3a0 0x7fce5c0de830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51474/0 (socket says 172.21.15.40:51474) 2024-04-25T11:38:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce62d9d640 1 -- 172.21.15.40:0/3771522647 learned_addr learned my addr 172.21.15.40:0/3771522647 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce61d9b640 1 --2- 172.21.15.40:0/3771522647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c045710 0x7fce5c045b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7f0d6bfff640 1 --2- 172.21.15.40:0/837088096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d6e0 0x7f0d6c06ca80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c15cce0 tx=0x7f0d6402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea563b245d34888f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7fa11affd640 1 --2- 172.21.15.40:0/2086588747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c15f160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa114009e50 tx=0x7fa11402a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03cf2c281db6bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7f3f80d95640 1 Processor -- start 2024-04-25T11:38:32.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7f3f80d95640 1 -- start start 2024-04-25T11:38:32.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce62d9d640 1 -- 172.21.15.40:0/3771522647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c045710 msgr2=0x7fce5c045b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce62d9d640 1 --2- 172.21.15.40:0/3771522647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c045710 0x7fce5c045b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce62d9d640 1 -- 172.21.15.40:0/3771522647 >> v1:172.21.15.40:6789/0 conn(0x7fce5c0d5080 legacy=0x7fce5c04b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.263+0000 7fce62d9d640 1 -- 172.21.15.40:0/3771522647 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c007490 con 0x7fce5c0de3a0 2024-04-25T11:38:32.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7f0d6b7fe640 1 -- 172.21.15.40:0/837088096 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64036070 con 0x7f0d6c06d6e0 2024-04-25T11:38:32.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7f4d8f59e640 1 Processor -- start 2024-04-25T11:38:32.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7f4d8f59e640 1 -- start start 2024-04-25T11:38:32.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7f0d6b7fe640 1 -- 172.21.15.40:0/837088096 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d64031600 con 0x7f0d6c06d6e0 2024-04-25T11:38:32.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7fa11a7fc640 1 -- 172.21.15.40:0/2086588747 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114036070 con 0x7fa11c0cb120 2024-04-25T11:38:32.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7fa11a7fc640 1 -- 172.21.15.40:0/2086588747 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa114031600 con 0x7fa11c0cb120 2024-04-25T11:38:32.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.264+0000 7fa11a7fc640 1 -- 172.21.15.40:0/2086588747 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1140319d0 con 0x7fa11c0cb120 2024-04-25T11:38:32.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7fce62d9d640 1 --2- 172.21.15.40:0/3771522647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5c0de3a0 0x7fce5c0de830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fce64065320 tx=0x7fce640a6400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e30e24d1def19418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7f0d6b7fe640 1 -- 172.21.15.40:0/837088096 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d640319d0 con 0x7f0d6c06d6e0 2024-04-25T11:38:32.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7fce6159a640 1 -- 172.21.15.40:0/3771522647 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640b2070 con 0x7fce5c0de3a0 2024-04-25T11:38:32.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7fce6159a640 1 -- 172.21.15.40:0/3771522647 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce640ab7d0 con 0x7fce5c0de3a0 2024-04-25T11:38:32.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7fce6159a640 1 -- 172.21.15.40:0/3771522647 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640abc40 con 0x7fce5c0de3a0 2024-04-25T11:38:32.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7f0d7323c640 1 -- 172.21.15.40:0/837088096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d6e0 msgr2=0x7f0d6c06ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7f3f80d95640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3f7c0c89f0 0x7f3f7c0c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7f3f80d95640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f7c072710 0x7f3f7c072b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7fa1220e1640 1 -- 172.21.15.40:0/2086588747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 msgr2=0x7fa11c15f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7fa1220e1640 1 --2- 172.21.15.40:0/2086588747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c15f160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa114009e50 tx=0x7fa11402a760 comp rx=0 tx=0).stop 2024-04-25T11:38:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f3f80d95640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f7c0c3620 con 0x7f3f7c1519f0 2024-04-25T11:38:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7fae69a43640 1 Processor -- start 2024-04-25T11:38:32.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7f0d7323c640 1 --2- 172.21.15.40:0/837088096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d6e0 0x7f0d6c06ca80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c15cce0 tx=0x7f0d6402a710 comp rx=0 tx=0).stop 2024-04-25T11:38:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f3f80d95640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3f7c06fa00 con 0x7f3f7c0c89f0 2024-04-25T11:38:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f3f80d95640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f3f7c151dd0 con 0x7f3f7c072710 2024-04-25T11:38:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7f925a130640 1 Processor -- start 2024-04-25T11:38:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7f925a130640 1 -- start start 2024-04-25T11:38:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f3f7ad9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f7c072710 0x7f3f7c072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f3f7ad9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f7c072710 0x7f3f7c072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51490/0 (socket says 172.21.15.40:51490) 2024-04-25T11:38:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7f4d8f59e640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006bbe0 0x7f4d900c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f4d8f59e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4d9006bfc0 0x7f4d90153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f4d8f59e640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d90156bb0 con 0x7f4d9006e660 2024-04-25T11:38:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f4d8f59e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4d900c7990 con 0x7f4d9006bfc0 2024-04-25T11:38:32.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f4d8f59e640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4d900c7c70 con 0x7f4d9006bbe0 2024-04-25T11:38:32.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.265+0000 7f5f0535b640 1 --2- 172.21.15.40:0/3648975492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef80d5200 0x7f5ef804b620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5efc008870 tx=0x7f5efc02c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3815b509d1eb2fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f5ef7fff640 1 -- 172.21.15.40:0/3648975492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc03a070 con 0x7f5ef80d5200 2024-04-25T11:38:32.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f5ef7fff640 1 -- 172.21.15.40:0/3648975492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5efc032480 con 0x7f5ef80d5200 2024-04-25T11:38:32.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f5ef7fff640 1 -- 172.21.15.40:0/3648975492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc032850 con 0x7f5ef80d5200 2024-04-25T11:38:32.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f3f7ad9d640 1 -- 172.21.15.40:0/3817021979 learned_addr learned my addr 172.21.15.40:0/3817021979 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7fae69a43640 1 -- start start 2024-04-25T11:38:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7fce68d6f640 1 -- 172.21.15.40:0/3771522647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5c0de3a0 msgr2=0x7fce5c0de830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7fce68d6f640 1 --2- 172.21.15.40:0/3771522647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5c0de3a0 0x7fce5c0de830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fce64065320 tx=0x7fce640a6400 comp rx=0 tx=0).stop 2024-04-25T11:38:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f925a130640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9254151220 0x7f9254072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f925a130640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9254157da0 con 0x7f9254072900 2024-04-25T11:38:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f925a130640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925406eaf0 con 0x7f925406c490 2024-04-25T11:38:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f925a130640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f9254157500 con 0x7f9254151220 2024-04-25T11:38:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f3f79d9b640 1 --2- 172.21.15.40:0/3817021979 >> v2:172.21.15.40:3300/0 conn(0x7f3f7c0c89f0 0x7f3f7c0c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f9253fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9254072900 0x7f92540c8d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43932/0 (socket says 172.21.15.40:43932) 2024-04-25T11:38:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f9253fff640 1 -- 172.21.15.40:0/646838757 learned_addr learned my addr 172.21.15.40:0/646838757 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f9252ffd640 1 --2- 172.21.15.40:0/646838757 >> v2:172.21.15.62:3300/0 conn(0x7f9254151220 0x7f9254072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f3f7959a640 1 -- 172.21.15.40:0/3817021979 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 41780658 0 0) 0x7f3f6c001110 con 0x7f3f7c1519f0 2024-04-25T11:38:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f3f7959a640 1 -- 172.21.15.40:0/3817021979 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f64003520 con 0x7f3f7c1519f0 2024-04-25T11:38:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fce68d6f640 1 -- 172.21.15.40:0/3771522647 shutdown_connections 2024-04-25T11:38:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fce68d6f640 1 --2- 172.21.15.40:0/3771522647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5c0de3a0 0x7fce5c0de830 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fce68d6f640 1 --2- 172.21.15.40:0/3771522647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c045710 0x7fce5c045b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fce68d6f640 1 -- 172.21.15.40:0/3771522647 >> 172.21.15.40:0/3771522647 conn(0x7fce5c0505c0 msgr2=0x7fce5c0509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f9252ffd640 1 -- 172.21.15.40:0/646838757 >> v1:172.21.15.62:6789/0 conn(0x7f925406c490 legacy=0x7f92540c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f9252ffd640 1 -- 172.21.15.40:0/646838757 >> v1:172.21.15.40:6789/0 conn(0x7f9254072900 legacy=0x7f92540c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f9252ffd640 1 -- 172.21.15.40:0/646838757 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92541577e0 con 0x7f9254151220 2024-04-25T11:38:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f5f0635d640 1 -- 172.21.15.40:0/3648975492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef80d5200 msgr2=0x7f5ef804b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f5f0635d640 1 --2- 172.21.15.40:0/3648975492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef80d5200 0x7f5ef804b620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5efc008870 tx=0x7f5efc02c7a0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f4d8e59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006bbe0 0x7f4d900c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f4d8e59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006bbe0 0x7f4d900c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40708/0 (socket says 172.21.15.40:40708) 2024-04-25T11:38:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.266+0000 7f4d8e59c640 1 -- 172.21.15.40:0/3848886667 learned_addr learned my addr 172.21.15.40:0/3848886667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f4d8e59c640 1 -- 172.21.15.40:0/3848886667 >> v2:172.21.15.40:3300/0 conn(0x7f4d9006bfc0 msgr2=0x7f4d90153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f4d8e59c640 1 --2- 172.21.15.40:0/3848886667 >> v2:172.21.15.40:3300/0 conn(0x7f4d9006bfc0 0x7f4d90153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f4d8e59c640 1 -- 172.21.15.40:0/3848886667 >> v1:172.21.15.62:6789/0 conn(0x7f4d9006e660 legacy=0x7f4d90157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f4d8e59c640 1 -- 172.21.15.40:0/3848886667 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d900c7f50 con 0x7f4d9006bbe0 2024-04-25T11:38:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f5f0635d640 1 -- 172.21.15.40:0/3648975492 shutdown_connections 2024-04-25T11:38:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f5f0635d640 1 --2- 172.21.15.40:0/3648975492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ef804b090 0x7f5ef80db830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f5f0635d640 1 --2- 172.21.15.40:0/3648975492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef80d5200 0x7f5ef804b620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f5f0635d640 1 -- 172.21.15.40:0/3648975492 >> 172.21.15.40:0/3648975492 conn(0x7f5ef8050180 msgr2=0x7f5ef8050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fce68d6f640 1 -- 172.21.15.40:0/3771522647 shutdown_connections 2024-04-25T11:38:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fae69a43640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fae640ce640 0x7fae64154960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fae69a43640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae64070f70 0x7fae64154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fae69a43640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae6415ed00 con 0x7fae64070b90 2024-04-25T11:38:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fae69a43640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fae6406b970 con 0x7fae64070f70 2024-04-25T11:38:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fae69a43640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fae6415efa0 con 0x7fae640ce640 2024-04-25T11:38:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f5f0635d640 1 -- 172.21.15.40:0/3648975492 shutdown_connections 2024-04-25T11:38:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fa1220e1640 1 -- 172.21.15.40:0/2086588747 shutdown_connections 2024-04-25T11:38:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fa1220e1640 1 --2- 172.21.15.40:0/2086588747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c15f160 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fa1220e1640 1 --2- 172.21.15.40:0/2086588747 >> v2:172.21.15.62:3300/0 conn(0x7fa11c1595d0 0x7fa11c1605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fa1220e1640 1 -- 172.21.15.40:0/2086588747 >> 172.21.15.40:0/2086588747 conn(0x7fa11c073be0 msgr2=0x7fa11c06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f5f0635d640 1 -- 172.21.15.40:0/3648975492 wait complete. 2024-04-25T11:38:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f3f7ad9d640 1 -- 172.21.15.40:0/3817021979 >> v2:172.21.15.40:3300/0 conn(0x7f3f7c0c89f0 msgr2=0x7f3f7c0c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f3f7ad9d640 1 --2- 172.21.15.40:0/3817021979 >> v2:172.21.15.40:3300/0 conn(0x7f3f7c0c89f0 0x7f3f7c0c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f3f7ad9d640 1 -- 172.21.15.40:0/3817021979 >> v1:172.21.15.62:6789/0 conn(0x7f3f7c1519f0 legacy=0x7f3f7c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7f3f7ad9d640 1 -- 172.21.15.40:0/3817021979 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c15b2e0 con 0x7f3f7c072710 2024-04-25T11:38:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fae627fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fae640ce640 0x7fae64154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fae62ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fae64070b90 0x7fae640cdad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43942/0 (socket says 172.21.15.40:43942) 2024-04-25T11:38:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fae62ffd640 1 -- 172.21.15.40:0/692979847 learned_addr learned my addr 172.21.15.40:0/692979847 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fae63fff640 1 --2- 172.21.15.40:0/692979847 >> v2:172.21.15.40:3300/0 conn(0x7fae64070f70 0x7fae64154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f0d7323c640 1 -- 172.21.15.40:0/837088096 shutdown_connections 2024-04-25T11:38:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f0d7323c640 1 --2- 172.21.15.40:0/837088096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d6e0 0x7f0d6c06ca80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7fa1220e1640 1 -- 172.21.15.40:0/2086588747 shutdown_connections 2024-04-25T11:38:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.267+0000 7fce68d6f640 1 -- 172.21.15.40:0/3771522647 wait complete. 2024-04-25T11:38:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f0d7323c640 1 --2- 172.21.15.40:0/837088096 >> v2:172.21.15.62:3300/0 conn(0x7f0d6c06d300 0x7f0d6c15fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f0d7323c640 1 -- 172.21.15.40:0/837088096 >> 172.21.15.40:0/837088096 conn(0x7f0d6c0713c0 msgr2=0x7f0d6c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7fa1220e1640 1 -- 172.21.15.40:0/2086588747 wait complete. 2024-04-25T11:38:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f3f7ad9d640 1 --2- 172.21.15.40:0/3817021979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c072710 0x7f3f7c072b60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0c4550 tx=0x7f3f7402ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aada88ab371dbedf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f9252ffd640 1 --2- 172.21.15.40:0/646838757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254151220 0x7f9254072ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f924400ba40 tx=0x7f924402c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87746352367a0230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f0d7323c640 1 -- 172.21.15.40:0/837088096 shutdown_connections 2024-04-25T11:38:32.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f3f7959a640 1 -- 172.21.15.40:0/3817021979 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74035020 con 0x7f3f7c072710 2024-04-25T11:38:32.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f3f7959a640 1 -- 172.21.15.40:0/3817021979 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f74009600 con 0x7f3f7c072710 2024-04-25T11:38:32.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f3f7959a640 1 -- 172.21.15.40:0/3817021979 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74009a10 con 0x7f3f7c072710 2024-04-25T11:38:32.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7fae627fc640 1 -- 172.21.15.40:0/692979847 >> v2:172.21.15.40:3300/0 conn(0x7fae64070f70 msgr2=0x7fae64154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7fae627fc640 1 --2- 172.21.15.40:0/692979847 >> v2:172.21.15.40:3300/0 conn(0x7fae64070f70 0x7fae64154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7fae61ffb640 1 -- 172.21.15.40:0/692979847 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 947731516 0 0) 0x7fae6415ed00 con 0x7fae64070b90 2024-04-25T11:38:32.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7fae627fc640 1 -- 172.21.15.40:0/692979847 >> v1:172.21.15.40:6789/0 conn(0x7fae64070b90 legacy=0x7fae640cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7fae627fc640 1 -- 172.21.15.40:0/692979847 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae640ccd10 con 0x7fae640ce640 2024-04-25T11:38:32.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f5f0635d640 1 Processor -- start 2024-04-25T11:38:32.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f3f80d95640 1 -- 172.21.15.40:0/3817021979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c072710 msgr2=0x7f3f7c072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f5f0635d640 1 -- start start 2024-04-25T11:38:32.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f92527fc640 1 -- 172.21.15.40:0/646838757 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244032750 con 0x7f9254151220 2024-04-25T11:38:32.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f0d7323c640 1 -- 172.21.15.40:0/837088096 wait complete. 2024-04-25T11:38:32.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f92527fc640 1 -- 172.21.15.40:0/646838757 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9244030050 con 0x7f9254151220 2024-04-25T11:38:32.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.268+0000 7f92527fc640 1 -- 172.21.15.40:0/646838757 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244032b10 con 0x7f9254151220 2024-04-25T11:38:32.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f925a130640 1 -- 172.21.15.40:0/646838757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254151220 msgr2=0x7f9254072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f925a130640 1 --2- 172.21.15.40:0/646838757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254151220 0x7f9254072ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f924400ba40 tx=0x7f924402c850 comp rx=0 tx=0).stop 2024-04-25T11:38:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f4d8e59c640 1 --2- 172.21.15.40:0/3848886667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006bbe0 0x7f4d900c6ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4d88007a60 tx=0x7f4d88032ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44f6a932280bd39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f4d8d59a640 1 -- 172.21.15.40:0/3848886667 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88039070 con 0x7f4d9006bbe0 2024-04-25T11:38:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f4d8d59a640 1 -- 172.21.15.40:0/3848886667 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d88033d00 con 0x7f4d9006bbe0 2024-04-25T11:38:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f4d8d59a640 1 -- 172.21.15.40:0/3848886667 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88039070 con 0x7f4d9006bbe0 2024-04-25T11:38:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7fce68d6f640 1 Processor -- start 2024-04-25T11:38:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7fce68d6f640 1 -- start start 2024-04-25T11:38:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f3f80d95640 1 --2- 172.21.15.40:0/3817021979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c072710 0x7f3f7c072b60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0c4550 tx=0x7f3f7402ca20 comp rx=0 tx=0).stop 2024-04-25T11:38:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f3f80d95640 1 -- 172.21.15.40:0/3817021979 shutdown_connections 2024-04-25T11:38:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f3f80d95640 1 --2- 172.21.15.40:0/3817021979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c072710 0x7f3f7c072b60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f3f80d95640 1 --2- 172.21.15.40:0/3817021979 >> v2:172.21.15.40:3300/0 conn(0x7f3f7c0c89f0 0x7f3f7c0c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f3f80d95640 1 -- 172.21.15.40:0/3817021979 >> 172.21.15.40:0/3817021979 conn(0x7f3f7c06d2e0 msgr2=0x7f3f7c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7fa1220e1640 1 Processor -- start 2024-04-25T11:38:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f925a130640 1 -- 172.21.15.40:0/646838757 shutdown_connections 2024-04-25T11:38:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7f925a130640 1 --2- 172.21.15.40:0/646838757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254151220 0x7f9254072ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f925a130640 1 -- 172.21.15.40:0/646838757 >> 172.21.15.40:0/646838757 conn(0x7f925406b970 msgr2=0x7f925406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.269+0000 7fae627fc640 1 --2- 172.21.15.40:0/692979847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae640ce640 0x7fae64154960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae64068220 tx=0x7fae4c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b947c3d4c7f765d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7fae61ffb640 1 -- 172.21.15.40:0/692979847 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c037070 con 0x7fae640ce640 2024-04-25T11:38:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7fae61ffb640 1 -- 172.21.15.40:0/692979847 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4c032440 con 0x7fae640ce640 2024-04-25T11:38:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7fae61ffb640 1 -- 172.21.15.40:0/692979847 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0327f0 con 0x7fae640ce640 2024-04-25T11:38:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7fa1220e1640 1 -- start start 2024-04-25T11:38:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3848886667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006bbe0 msgr2=0x7f4d900c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f4d8f59e640 1 --2- 172.21.15.40:0/3848886667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006bbe0 0x7f4d900c6ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4d88007a60 tx=0x7f4d88032ff0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f925a130640 1 -- 172.21.15.40:0/646838757 shutdown_connections 2024-04-25T11:38:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f925a130640 1 -- 172.21.15.40:0/646838757 wait complete. 2024-04-25T11:38:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f0d7323c640 1 Processor -- start 2024-04-25T11:38:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7fce68d6f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce5c045710 0x7fce5c0d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7fce68d6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5c0d5080 0x7fce5c0dd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f5f0635d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ef8005ee0 0x7f5ef80c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7fce68d6f640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c0de3a0 0x7fce5c0d6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f5f0635d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef804b090 0x7f5ef80d3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f5f0635d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ef80d5200 0x7f5ef80de600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f0635d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8046a10 con 0x7f5ef804b090 2024-04-25T11:38:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f0635d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8046730 con 0x7f5ef80d5200 2024-04-25T11:38:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce68d6f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0d5460 con 0x7fce5c045710 2024-04-25T11:38:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f0635d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef8045a40 con 0x7f5ef8005ee0 2024-04-25T11:38:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce68d6f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c046050 con 0x7fce5c0d5080 2024-04-25T11:38:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce68d6f640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c04cb40 con 0x7fce5c0de3a0 2024-04-25T11:38:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce61d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5c0d5080 0x7fce5c0dd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce6259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce5c045710 0x7fce5c0d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce62d9d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c0de3a0 0x7fce5c0d6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce61d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5c0d5080 0x7fce5c0dd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51506/0 (socket says 172.21.15.40:51506) 2024-04-25T11:38:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f05b5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ef80d5200 0x7f5ef80de600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f05b5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ef80d5200 0x7f5ef80de600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51508/0 (socket says 172.21.15.40:51508) 2024-04-25T11:38:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f05b5c640 1 -- 172.21.15.40:0/4003648555 learned_addr learned my addr 172.21.15.40:0/4003648555 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fa1220e1640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f04b5a640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef804b090 0x7f5ef80d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fa1220e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa11c1595d0 0x7fa11c15d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f0d7323c640 1 -- start start 2024-04-25T11:38:32.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce61d9b640 1 -- 172.21.15.40:0/3833334988 learned_addr learned my addr 172.21.15.40:0/3833334988 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce6259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce5c045710 0x7fce5c0d8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57584/0 (socket says 172.21.15.40:57584) 2024-04-25T11:38:32.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fa11b7fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fce62d9d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c0de3a0 0x7fce5c0d6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40732/0 (socket says 172.21.15.40:40732) 2024-04-25T11:38:32.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fae69a43640 1 -- 172.21.15.40:0/692979847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae640ce640 msgr2=0x7fae64154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fae69a43640 1 --2- 172.21.15.40:0/692979847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae640ce640 0x7fae64154960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae64068220 tx=0x7fae4c02a250 comp rx=0 tx=0).stop 2024-04-25T11:38:32.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f05b5c640 1 -- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ef8005ee0 msgr2=0x7f5ef80c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f0535b640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ef8005ee0 0x7f5ef80c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f05b5c640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ef8005ee0 0x7f5ef80c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f05b5c640 1 -- 172.21.15.40:0/4003648555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef804b090 msgr2=0x7f5ef80d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f05b5c640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef804b090 0x7f5ef80d3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f5f05b5c640 1 -- 172.21.15.40:0/4003648555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef80caa90 con 0x7f5ef80d5200 2024-04-25T11:38:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.270+0000 7f3f80d95640 1 -- 172.21.15.40:0/3817021979 shutdown_connections 2024-04-25T11:38:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f3f80d95640 1 -- 172.21.15.40:0/3817021979 wait complete. 2024-04-25T11:38:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3848886667 shutdown_connections 2024-04-25T11:38:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f4d8f59e640 1 --2- 172.21.15.40:0/3848886667 >> v2:172.21.15.40:3300/0 conn(0x7f4d9006bfc0 0x7f4d90153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fae69a43640 1 -- 172.21.15.40:0/692979847 shutdown_connections 2024-04-25T11:38:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f4d8f59e640 1 --2- 172.21.15.40:0/3848886667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006bbe0 0x7f4d900c6ad0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3848886667 >> 172.21.15.40:0/3848886667 conn(0x7f4d900cc0d0 msgr2=0x7f4d900cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fae69a43640 1 --2- 172.21.15.40:0/692979847 >> v2:172.21.15.40:3300/0 conn(0x7fae64070f70 0x7fae64154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fae69a43640 1 --2- 172.21.15.40:0/692979847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae640ce640 0x7fae64154960 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae64068220 tx=0x7fae4c02a250 comp rx=0 tx=0).stop 2024-04-25T11:38:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7f5f05b5c640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ef80d5200 0x7f5ef80de600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000b490 tx=0x7f5ef000b950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fae69a43640 1 -- 172.21.15.40:0/692979847 >> 172.21.15.40:0/692979847 conn(0x7fae64067890 msgr2=0x7fae64072d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fce62d9d640 1 -- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5c0d5080 msgr2=0x7fce5c0dd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fce62d9d640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5c0d5080 0x7fce5c0dd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3848886667 shutdown_connections 2024-04-25T11:38:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3848886667 wait complete. 2024-04-25T11:38:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fce62d9d640 1 -- 172.21.15.40:0/3833334988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce5c045710 msgr2=0x7fce5c0d8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fce62d9d640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce5c045710 0x7fce5c0d8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fa11b7fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40746/0 (socket says 172.21.15.40:40746) 2024-04-25T11:38:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.271+0000 7fa11b7fe640 1 -- 172.21.15.40:0/2001097725 learned_addr learned my addr 172.21.15.40:0/2001097725 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa11affd640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa11c1595d0 0x7fa11c15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa1220e1640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa11c15f6a0 0x7fa11c149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c160b10 con 0x7fa11c15f6a0 2024-04-25T11:38:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c162c00 con 0x7fa11c1595d0 2024-04-25T11:38:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c06d190 con 0x7fa11c0cb120 2024-04-25T11:38:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fce62d9d640 1 -- 172.21.15.40:0/3833334988 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c0349d0 con 0x7fce5c0de3a0 2024-04-25T11:38:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7f925a130640 1 Processor -- start 2024-04-25T11:38:32.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fce6259c640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce5c045710 0x7fce5c0d8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:32.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa11bfff640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa11c15f6a0 0x7fa11c149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fae69a43640 1 -- 172.21.15.40:0/692979847 shutdown_connections 2024-04-25T11:38:32.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7f925a130640 1 -- start start 2024-04-25T11:38:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa11b7fe640 1 -- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa11c1595d0 msgr2=0x7fa11c15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa11b7fe640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa11c1595d0 0x7fa11c15d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa11b7fe640 1 -- 172.21.15.40:0/2001097725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa11c15f6a0 msgr2=0x7fa11c149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa11b7fe640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa11c15f6a0 0x7fa11c149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fa11b7fe640 1 -- 172.21.15.40:0/2001097725 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c144ea0 con 0x7fa11c0cb120 2024-04-25T11:38:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7f5ef5ffb640 1 -- 172.21.15.40:0/4003648555 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0015020 con 0x7f5ef80d5200 2024-04-25T11:38:32.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7f5ef5ffb640 1 -- 172.21.15.40:0/4003648555 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef0011850 con 0x7f5ef80d5200 2024-04-25T11:38:32.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7f5ef5ffb640 1 -- 172.21.15.40:0/4003648555 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0011c40 con 0x7f5ef80d5200 2024-04-25T11:38:32.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7f0d7323c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d300 0x7f0d6c1f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7fa11b7fe640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c149370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00e9e0 tx=0x7fa10c00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef80dd840 con 0x7f5ef80d5200 2024-04-25T11:38:32.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7fa103fff640 1 -- 172.21.15.40:0/2001097725 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c017070 con 0x7fa11c0cb120 2024-04-25T11:38:32.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7fa103fff640 1 -- 172.21.15.40:0/2001097725 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10c00c9d0 con 0x7fa11c0cb120 2024-04-25T11:38:32.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c144bc0 con 0x7fa11c0cb120 2024-04-25T11:38:32.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7f925a130640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406c490 0x7f925414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef80dde00 con 0x7f5ef80d5200 2024-04-25T11:38:32.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f4d8f59e640 1 Processor -- start 2024-04-25T11:38:32.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11c146c50 con 0x7fa11c0cb120 2024-04-25T11:38:32.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f925a130640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254072900 0x7f9254152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f3f80d95640 1 Processor -- start 2024-04-25T11:38:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f925a130640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254151220 0x7f92541f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7fce62d9d640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c0de3a0 0x7fce5c0d6e80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fce6407bc70 tx=0x7fce64063750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f3f80d95640 1 -- start start 2024-04-25T11:38:32.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f925a130640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92540c7a90 con 0x7f9254072900 2024-04-25T11:38:32.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7fce5affd640 1 -- 172.21.15.40:0/3833334988 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640bd020 con 0x7fce5c0de3a0 2024-04-25T11:38:32.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f9252ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254072900 0x7f9254152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f9252ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254072900 0x7f9254152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57628/0 (socket says 172.21.15.40:57628) 2024-04-25T11:38:32.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f9252ffd640 1 -- 172.21.15.40:0/1364971720 learned_addr learned my addr 172.21.15.40:0/1364971720 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f4d8f59e640 1 -- start start 2024-04-25T11:38:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92540c77b0 con 0x7f9254151220 2024-04-25T11:38:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f92540c86c0 con 0x7f925406c490 2024-04-25T11:38:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f92537fe640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406c490 0x7f925414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.272+0000 7fae69a43640 1 -- 172.21.15.40:0/692979847 wait complete. 2024-04-25T11:38:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7fae69a43640 1 Processor -- start 2024-04-25T11:38:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.273+0000 7fce5affd640 1 -- 172.21.15.40:0/3833334988 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce640ada30 con 0x7fce5c0de3a0 2024-04-25T11:38:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f9253fff640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254151220 0x7f92541f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7fce5affd640 1 -- 172.21.15.40:0/3833334988 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640b2040 con 0x7fce5c0de3a0 2024-04-25T11:38:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7fae69a43640 1 -- start start 2024-04-25T11:38:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f0d70fd8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d300 0x7f0d6c1f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7fce68d6f640 1 -- 172.21.15.40:0/3833334988 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c034cb0 con 0x7fce5c0de3a0 2024-04-25T11:38:32.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f0d7323c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d6c06d6e0 0x7f0d6c161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7fce68d6f640 1 -- 172.21.15.40:0/3833334988 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c035270 con 0x7fce5c0de3a0 2024-04-25T11:38:32.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f9253fff640 1 -- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406c490 msgr2=0x7f925414ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d9006bbe0 0x7f4d901563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d9006bfc0 0x7f4d901f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8f59e640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006e660 0x7f4d900c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f9253fff640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406c490 0x7f925414ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f9253fff640 1 -- 172.21.15.40:0/1364971720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254072900 msgr2=0x7f9254152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f9253fff640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254072900 0x7f9254152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f3f80d95640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f7c072710 0x7f3f7c1e3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f9253fff640 1 -- 172.21.15.40:0/1364971720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9254151af0 con 0x7f9254151220 2024-04-25T11:38:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4d900715a0 con 0x7f4d9006bbe0 2024-04-25T11:38:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8f59e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d900c8510 con 0x7f4d9006bfc0 2024-04-25T11:38:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d9006bbe0 0x7f4d901563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8f59e640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4d90071720 con 0x7f4d9006e660 2024-04-25T11:38:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d9006bbe0 0x7f4d901563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57640/0 (socket says 172.21.15.40:57640) 2024-04-25T11:38:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8e59c640 1 -- 172.21.15.40:0/3338584048 learned_addr learned my addr 172.21.15.40:0/3338584048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8dd9b640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d9006bfc0 0x7f4d901f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.275+0000 7f4d8ed9d640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006e660 0x7f4d900c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f0d7323c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d6c0c3c60 0x7f0d6c15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.274+0000 7f0d7323c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c06fe00 con 0x7f0d6c06d300 2024-04-25T11:38:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f0d70fd8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d300 0x7f0d6c1f98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57624/0 (socket says 172.21.15.40:57624) 2024-04-25T11:38:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f0d70fd8640 1 -- 172.21.15.40:0/3711069905 learned_addr learned my addr 172.21.15.40:0/3711069905 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c070450 con 0x7f0d6c06d6e0 2024-04-25T11:38:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0d6c0700e0 con 0x7f0d6c0c3c60 2024-04-25T11:38:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f0d6bfff640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d6c06d6e0 0x7f0d6c161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fce58ff9640 1 -- 172.21.15.40:0/3833334988 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce3c001030 con 0x7fce5c0de3a0 2024-04-25T11:38:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ef80052c0 con 0x7f5ef80d5200 2024-04-25T11:38:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f5ef5ffb640 1 -- 172.21.15.40:0/4003648555 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ef0018020 con 0x7f5ef80d5200 2024-04-25T11:38:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f4d8dd9b640 1 -- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006e660 msgr2=0x7f4d900c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f4d8dd9b640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d9006e660 0x7f4d900c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f4d8dd9b640 1 -- 172.21.15.40:0/3338584048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d9006bbe0 msgr2=0x7f4d901563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f4d8dd9b640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d9006bbe0 0x7f4d901563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f4d8dd9b640 1 -- 172.21.15.40:0/3338584048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d901ec540 con 0x7f4d9006bfc0 2024-04-25T11:38:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fa103fff640 1 -- 172.21.15.40:0/2001097725 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c00cdc0 con 0x7fa11c0cb120 2024-04-25T11:38:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f9252ffd640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254072900 0x7f9254152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f92537fe640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406c490 0x7f925414ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f9253fff640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254151220 0x7f92541f8e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f924800c390 tx=0x7f924800c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae69a43640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae64070b90 0x7fae640d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae69a43640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae64070f70 0x7fae6415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae69a43640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae641fe050 0x7fae6415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0fc001030 con 0x7fa11c0cb120 2024-04-25T11:38:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae627fc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae64070f70 0x7fae6415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae627fc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae64070f70 0x7fae6415cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40790/0 (socket says 172.21.15.40:40790) 2024-04-25T11:38:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae627fc640 1 -- 172.21.15.40:0/1426402920 learned_addr learned my addr 172.21.15.40:0/1426402920 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f923bfff640 1 -- 172.21.15.40:0/1364971720 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248017070 con 0x7f9254151220 2024-04-25T11:38:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fae640cd5b0 con 0x7fae641fe050 2024-04-25T11:38:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f0d717d9640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d6c0c3c60 0x7f0d6c15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae62ffd640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae64070b90 0x7fae640d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f3f80d95640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f7c0c89f0 0x7f3f7c1ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f3f80d95640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c1519f0 0x7f3f7c1e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f3f80d95640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c06f3b0 con 0x7f3f7c072710 2024-04-25T11:38:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f3f7a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f7c072710 0x7f3f7c1e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f3f7a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f7c072710 0x7f3f7c1e3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57652/0 (socket says 172.21.15.40:57652) 2024-04-25T11:38:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f3f7a59c640 1 -- 172.21.15.40:0/2149636270 learned_addr learned my addr 172.21.15.40:0/2149636270 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f3f80d95640 1 -- 172.21.15.40:0/2149636270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c06f0d0 con 0x7f3f7c1519f0 2024-04-25T11:38:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f3f80d95640 1 -- 172.21.15.40:0/2149636270 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7c06f690 con 0x7f3f7c0c89f0 2024-04-25T11:38:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae640cd2d0 con 0x7fae64070b90 2024-04-25T11:38:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fae640ce1e0 con 0x7fae64070f70 2024-04-25T11:38:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f923bfff640 1 -- 172.21.15.40:0/1364971720 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9248005ce0 con 0x7f9254151220 2024-04-25T11:38:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92540c04c0 con 0x7f9254151220 2024-04-25T11:38:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9254152090 con 0x7f9254151220 2024-04-25T11:38:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.276+0000 7f5ef5ffb640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5ee406c940 0x7f5ee406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f4d8dd9b640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d9006bfc0 0x7f4d901f0490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400b3f0 tx=0x7f4d8400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7fce5affd640 1 -- 172.21.15.40:0/3833334988 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce640c0020 con 0x7fce5c0de3a0 2024-04-25T11:38:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f5ef5ffb640 1 -- 172.21.15.40:0/4003648555 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5ef008b9c0 con 0x7f5ef80d5200 2024-04-25T11:38:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f5ef5ffb640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7f5ee4074bc0 0x7f5ee4076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f5ef5ffb640 1 -- 172.21.15.40:0/4003648555 --> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ee4077690 con 0x7f5ee4074bc0 2024-04-25T11:38:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f5ef5ffb640 1 -- 172.21.15.40:0/4003648555 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5ef00902c0 con 0x7f5ef80d5200 2024-04-25T11:38:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f3f79d9b640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f7c0c89f0 0x7f3f7c1ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f3f7a59c640 1 -- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f7c0c89f0 msgr2=0x7f3f7c1ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f3f7a59c640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f7c0c89f0 0x7f3f7c1ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f3f7a59c640 1 -- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c1519f0 msgr2=0x7f3f7c1e1f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:38:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f3f7a59c640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f7c1519f0 0x7f3f7c1e1f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f3f7a59c640 1 -- 172.21.15.40:0/2149636270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c1520d0 con 0x7f3f7c072710 2024-04-25T11:38:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7fce5affd640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fce4406c940 0x7fce4406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f923bfff640 1 -- 172.21.15.40:0/1364971720 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92480072d0 con 0x7f9254151220 2024-04-25T11:38:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fa103fff640 1 -- 172.21.15.40:0/2001097725 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa10c01a070 con 0x7fa11c0cb120 2024-04-25T11:38:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7fce5affd640 1 -- 172.21.15.40:0/3833334988 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce64128370 con 0x7fce5c0de3a0 2024-04-25T11:38:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f3f7a59c640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f7c072710 0x7f3f7c1e3610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00ea10 tx=0x7f3f6c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f0d70fd8640 1 -- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d6c0c3c60 msgr2=0x7f0d6c15dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f0d70fd8640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d6c0c3c60 0x7f0d6c15dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f0d70fd8640 1 -- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d6c06d6e0 msgr2=0x7f0d6c161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fce5affd640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7fce44074bc0 0x7fce44076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f5f0535b640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5ee406c940 0x7f5ee406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f0d70fd8640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d6c06d6e0 0x7f0d6c161960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f0d70fd8640 1 -- 172.21.15.40:0/3711069905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6c161ea0 con 0x7f0d6c06d300 2024-04-25T11:38:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fce5affd640 1 -- 172.21.15.40:0/3833334988 --> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce640758a0 con 0x7fce44074bc0 2024-04-25T11:38:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fce5affd640 1 -- 172.21.15.40:0/3833334988 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fce6412f050 con 0x7fce5c0de3a0 2024-04-25T11:38:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f4d7effd640 1 -- 172.21.15.40:0/3338584048 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84015020 con 0x7f4d9006bfc0 2024-04-25T11:38:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d901ec800 con 0x7f4d9006bfc0 2024-04-25T11:38:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.277+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d901ecdc0 con 0x7f4d9006bfc0 2024-04-25T11:38:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fce61d9b640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7fce44074bc0 0x7fce44076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fce6259c640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fce4406c940 0x7fce4406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f5f04b5a640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7f5ee4074bc0 0x7f5ee4076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fae63fff640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae641fe050 0x7fae6415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fae627fc640 1 -- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae64070b90 msgr2=0x7fae640d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fae627fc640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae64070b90 0x7fae640d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fae627fc640 1 -- 172.21.15.40:0/1426402920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae641fe050 msgr2=0x7fae6415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fae627fc640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae641fe050 0x7fae6415d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fae627fc640 1 -- 172.21.15.40:0/1426402920 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae640cfd00 con 0x7fae64070f70 2024-04-25T11:38:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f0d70fd8640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d300 0x7f0d6c1f98b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00b480 tx=0x7f0d5c00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f3f6affd640 1 -- 172.21.15.40:0/2149636270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c001cf0 con 0x7f3f7c072710 2024-04-25T11:38:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f0d697fa640 1 -- 172.21.15.40:0/3711069905 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c014070 con 0x7f0d6c06d300 2024-04-25T11:38:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7fa103fff640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa10806c9c0 0x7fa10806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f3f6affd640 1 -- 172.21.15.40:0/2149636270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0050d0 con 0x7f3f7c072710 2024-04-25T11:38:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f0d697fa640 1 -- 172.21.15.40:0/3711069905 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5c010890 con 0x7f0d6c06d300 2024-04-25T11:38:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f3f80d95640 1 -- 172.21.15.40:0/2149636270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c1523b0 con 0x7f3f7c072710 2024-04-25T11:38:32.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6c0b3d70 con 0x7f0d6c06d300 2024-04-25T11:38:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f5f0535b640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5ee406c940 0x7f5ee406ee00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5efc037000 tx=0x7f5efc00a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6c0b4010 con 0x7f0d6c06d300 2024-04-25T11:38:32.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7fce61d9b640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7fce44074bc0 0x7fce44076fe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f4d7effd640 1 -- 172.21.15.40:0/3338584048 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d84011830 con 0x7f4d9006bfc0 2024-04-25T11:38:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f4d7effd640 1 -- 172.21.15.40:0/3338584048 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84015020 con 0x7f4d9006bfc0 2024-04-25T11:38:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d74001030 con 0x7f4d9006bfc0 2024-04-25T11:38:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7fa103fff640 1 -- 172.21.15.40:0/2001097725 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa10c08ba10 con 0x7fa11c0cb120 2024-04-25T11:38:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7fae627fc640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae64070f70 0x7fae6415cc20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae4c000c00 tx=0x7fae4c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7fa103fff640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7fa108074c40 0x7fa108077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7fa103fff640 1 -- 172.21.15.40:0/2001097725 --> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa10c00c9d0 con 0x7fa108074c40 2024-04-25T11:38:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7fa103fff640 1 -- 172.21.15.40:0/2001097725 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa10c0902a0 con 0x7fa11c0cb120 2024-04-25T11:38:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f0d697fa640 1 -- 172.21.15.40:0/3711069905 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c010c80 con 0x7f0d6c06d300 2024-04-25T11:38:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f3f6affd640 1 -- 172.21.15.40:0/2149636270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0054a0 con 0x7f3f7c072710 2024-04-25T11:38:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f3f80d95640 1 -- 172.21.15.40:0/2149636270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c15ae70 con 0x7f3f7c072710 2024-04-25T11:38:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.278+0000 7f9239ffb640 1 -- 172.21.15.40:0/1364971720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f922c001030 con 0x7f9254151220 2024-04-25T11:38:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.279+0000 7f923bfff640 1 -- 172.21.15.40:0/1364971720 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f924801a070 con 0x7f9254151220 2024-04-25T11:38:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f923bfff640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f924006c9c0 0x7f924006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f923bfff640 1 -- 172.21.15.40:0/1364971720 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f924808ae80 con 0x7f9254151220 2024-04-25T11:38:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f923bfff640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f9240074c40 0x7f9240077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f923bfff640 1 -- 172.21.15.40:0/1364971720 --> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9240077710 con 0x7f9240074c40 2024-04-25T11:38:32.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f923bfff640 1 -- 172.21.15.40:0/1364971720 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f924808b270 con 0x7f9254151220 2024-04-25T11:38:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d6c1f5990 con 0x7f0d6c06d300 2024-04-25T11:38:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7fce5affd640 1 -- 172.21.15.40:0/3833334988 <== osd.1 v2:172.21.15.40:6800/3790556570 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce640758a0 con 0x7fce44074bc0 2024-04-25T11:38:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7fce6259c640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fce4406c940 0x7fce4406ee00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fce54007fd0 tx=0x7fce54007e20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f4d7effd640 1 -- 172.21.15.40:0/3338584048 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d84018020 con 0x7f4d9006bfc0 2024-04-25T11:38:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7fa11affd640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa10806c9c0 0x7fa10806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7fa11bfff640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7fa108074c40 0x7fa108077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7fa11affd640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa10806c9c0 0x7fa10806ee80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa114033000 tx=0x7fa114009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f9252ffd640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f9240074c40 0x7f9240077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f92537fe640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f924006c9c0 0x7f924006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7f9252ffd640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f9240074c40 0x7f9240077060 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7fae5bfff640 1 -- 172.21.15.40:0/1426402920 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c037040 con 0x7fae64070f70 2024-04-25T11:38:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7fae5bfff640 1 -- 172.21.15.40:0/1426402920 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4c042070 con 0x7fae64070f70 2024-04-25T11:38:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7fae5bfff640 1 -- 172.21.15.40:0/1426402920 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c006e10 con 0x7fae64070f70 2024-04-25T11:38:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.280+0000 7f3f68ff9640 1 -- 172.21.15.40:0/2149636270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f7c074c10 con 0x7f3f7c072710 2024-04-25T11:38:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae640cffe0 con 0x7fae64070f70 2024-04-25T11:38:32.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae640d0b30 con 0x7fae64070f70 2024-04-25T11:38:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7f4d7effd640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4d7006cb20 0x7f4d7006efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7f4d7effd640 1 -- 172.21.15.40:0/3338584048 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4d8408b7a0 con 0x7f4d9006bfc0 2024-04-25T11:38:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7f92537fe640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f924006c9c0 0x7f924006ee80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f924c001d70 tx=0x7f924c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7f4d8e59c640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4d7006cb20 0x7f4d7006efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7f4d7effd640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7f4d70074da0 0x7f4d700771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.282+0000 7f923bfff640 1 -- 172.21.15.40:0/1364971720 <== osd.6 v2:172.21.15.62:6802/1111875962 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9240077710 con 0x7f9240074c40 2024-04-25T11:38:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7f4d7effd640 1 -- 172.21.15.40:0/3338584048 --> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d84013ec0 con 0x7f4d70074da0 2024-04-25T11:38:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7f4d7effd640 1 -- 172.21.15.40:0/3338584048 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4d8408bb90 con 0x7f4d9006bfc0 2024-04-25T11:38:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.282+0000 7f4d8ed9d640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7f4d70074da0 0x7f4d700771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.281+0000 7f5f04b5a640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7f5ee4074bc0 0x7f5ee4076fe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.282+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae6404fdc0 con 0x7fae64070f70 2024-04-25T11:38:32.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.282+0000 7fa11bfff640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7fa108074c40 0x7fa108077060 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.282+0000 7f5ef5ffb640 1 -- 172.21.15.40:0/4003648555 <== osd.2 v2:172.21.15.40:6808/2321494678 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ee4077690 con 0x7f5ee4074bc0 2024-04-25T11:38:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.282+0000 7f4d8ed9d640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7f4d70074da0 0x7f4d700771c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.283+0000 7f4d8e59c640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4d7006cb20 0x7f4d7006efe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d88033030 tx=0x7f4d88039000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.283+0000 7f4d7effd640 1 -- 172.21.15.40:0/3338584048 <== osd.3 v2:172.21.15.40:6818/364049698 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d84013ec0 con 0x7f4d70074da0 2024-04-25T11:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.283+0000 7fae5bfff640 1 -- 172.21.15.40:0/1426402920 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae4c032830 con 0x7fae64070f70 2024-04-25T11:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.282+0000 7f3f6affd640 1 -- 172.21.15.40:0/2149636270 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f6c0035f0 con 0x7f3f7c072710 2024-04-25T11:38:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.283+0000 7f3f6affd640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3f5c06cb60 0x7f3f5c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.283+0000 7f3f6affd640 1 -- 172.21.15.40:0/2149636270 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f6c08f490 con 0x7f3f7c072710 2024-04-25T11:38:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.283+0000 7f3f79d9b640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3f5c06cb60 0x7f3f5c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.283+0000 7f3f6affd640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f3f5c074d60 0x7f3f5c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7f3f6affd640 1 -- 172.21.15.40:0/2149636270 --> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f6c003dc0 con 0x7f3f5c074d60 2024-04-25T11:38:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7f0d697fa640 1 -- 172.21.15.40:0/3711069905 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d5c017070 con 0x7f0d6c06d300 2024-04-25T11:38:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7fa103fff640 1 -- 172.21.15.40:0/2001097725 <== osd.5 v2:172.21.15.62:6824/3210426647 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa10c00c9d0 con 0x7fa108074c40 2024-04-25T11:38:32.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7fae5bfff640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae4406c9c0 0x7fae4406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7f3f6affd640 1 -- 172.21.15.40:0/2149636270 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3f6c096050 con 0x7f3f7c072710 2024-04-25T11:38:32.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7f0d697fa640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d5806c970 0x7f0d5806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7f0d697fa640 1 -- 172.21.15.40:0/3711069905 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0d5c08bc60 con 0x7f0d6c06d300 2024-04-25T11:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7f0d697fa640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f0d58074b90 0x7f0d58076fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7f0d697fa640 1 -- 172.21.15.40:0/3711069905 --> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d58077660 con 0x7f0d58074b90 2024-04-25T11:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.284+0000 7f0d697fa640 1 -- 172.21.15.40:0/3711069905 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0d5c08c140 con 0x7f0d6c06d300 2024-04-25T11:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7f3f7ad9d640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f3f5c074d60 0x7f3f5c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7f3f79d9b640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3f5c06cb60 0x7f3f5c06f020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c14d920 tx=0x7f3f70009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7fae5bfff640 1 -- 172.21.15.40:0/1426402920 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fae4c0ac1f0 con 0x7fae64070f70 2024-04-25T11:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7f0d6bfff640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d5806c970 0x7f0d5806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7f0d717d9640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f0d58074b90 0x7f0d58076fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7fae5bfff640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7fae44074c40 0x7fae44077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7fae5bfff640 1 -- 172.21.15.40:0/1426402920 --> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae4c032440 con 0x7fae44074c40 2024-04-25T11:38:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7fae5bfff640 1 -- 172.21.15.40:0/1426402920 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fae4c0b3050 con 0x7fae64070f70 2024-04-25T11:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7fae62ffd640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae4406c9c0 0x7fae4406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7fae63fff640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7fae44074c40 0x7fae44077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7f0d717d9640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f0d58074b90 0x7f0d58076fb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.285+0000 7f0d6bfff640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d5806c970 0x7f0d5806ee30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0d64033000 tx=0x7f0d64009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.286+0000 7f3f7ad9d640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f3f5c074d60 0x7f3f5c077180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.286+0000 7fae63fff640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7fae44074c40 0x7fae44077060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.286+0000 7fae62ffd640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae4406c9c0 0x7fae4406ee80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae54007f70 tx=0x7fae54007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.287+0000 7fae5bfff640 1 -- 172.21.15.40:0/1426402920 <== osd.7 v2:172.21.15.62:6816/3160032582 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fae4c032440 con 0x7fae44074c40 2024-04-25T11:38:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.287+0000 7f0d697fa640 1 -- 172.21.15.40:0/3711069905 <== osd.0 v2:172.21.15.40:6816/2617970798 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d58077660 con 0x7f0d58074b90 2024-04-25T11:38:32.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.288+0000 7f3f6affd640 1 -- 172.21.15.40:0/2149636270 <== osd.4 v2:172.21.15.62:6800/2239863089 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f6c003dc0 con 0x7f3f5c074d60 2024-04-25T11:38:32.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.310+0000 7fce58ff9640 1 -- 172.21.15.40:0/3833334988 --> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce3c002dd0 con 0x7fce44074bc0 2024-04-25T11:38:32.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.311+0000 7fce5affd640 1 -- 172.21.15.40:0/3833334988 <== osd.1 v2:172.21.15.40:6800/3790556570 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fce3c002dd0 con 0x7fce44074bc0 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.311+0000 7fce68d6f640 1 -- 172.21.15.40:0/3833334988 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7fce44074bc0 msgr2=0x7fce44076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.311+0000 7fce68d6f640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7fce44074bc0 0x7fce44076fe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.312+0000 7fce68d6f640 1 -- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fce4406c940 msgr2=0x7fce4406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.312+0000 7fce68d6f640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fce4406c940 0x7fce4406ee00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fce54007fd0 tx=0x7fce54007e20 comp rx=0 tx=0).stop 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.312+0000 7fce68d6f640 1 -- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c0de3a0 msgr2=0x7fce5c0d6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.312+0000 7fce68d6f640 1 --2- 172.21.15.40:0/3833334988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5c0de3a0 0x7fce5c0d6e80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fce6407bc70 tx=0x7fce64063750 comp rx=0 tx=0).stop 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.312+0000 7fce62d9d640 1 -- 172.21.15.40:0/3833334988 reap_dead start 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.312+0000 7fce68d6f640 1 -- 172.21.15.40:0/3833334988 shutdown_connections 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.312+0000 7fce68d6f640 1 -- 172.21.15.40:0/3833334988 >> 172.21.15.40:0/3833334988 conn(0x7fce5c0505c0 msgr2=0x7fce5c0d3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.312+0000 7fce68d6f640 1 -- 172.21.15.40:0/3833334988 shutdown_connections 2024-04-25T11:38:32.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.312+0000 7fce68d6f640 1 -- 172.21.15.40:0/3833334988 wait complete. 2024-04-25T11:38:32.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.314+0000 7f9239ffb640 1 -- 172.21.15.40:0/1364971720 --> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f922c002dd0 con 0x7f9240074c40 2024-04-25T11:38:32.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.315+0000 7f923bfff640 1 -- 172.21.15.40:0/1364971720 <== osd.6 v2:172.21.15.62:6802/1111875962 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f922c002dd0 con 0x7f9240074c40 2024-04-25T11:38:32.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.316+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 --> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ef80cb3d0 con 0x7f5ee4074bc0 2024-04-25T11:38:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.316+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f9240074c40 msgr2=0x7f9240077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.316+0000 7f925a130640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f9240074c40 0x7f9240077060 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.316+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f924006c9c0 msgr2=0x7f924006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.316+0000 7f925a130640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f924006c9c0 0x7f924006ee80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f924c001d70 tx=0x7f924c009040 comp rx=0 tx=0).stop 2024-04-25T11:38:32.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.316+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254151220 msgr2=0x7f92541f8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.316+0000 7f925a130640 1 --2- 172.21.15.40:0/1364971720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254151220 0x7f92541f8e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f924800c390 tx=0x7f924800c790 comp rx=0 tx=0).stop 2024-04-25T11:38:32.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.316+0000 7f9253fff640 1 -- 172.21.15.40:0/1364971720 reap_dead start 2024-04-25T11:38:32.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.317+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 shutdown_connections 2024-04-25T11:38:32.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.317+0000 7f5ef5ffb640 1 -- 172.21.15.40:0/4003648555 <== osd.2 v2:172.21.15.40:6808/2321494678 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5ef80cb3d0 con 0x7f5ee4074bc0 2024-04-25T11:38:32.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.317+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 >> 172.21.15.40:0/1364971720 conn(0x7f925406b970 msgr2=0x7f92540c8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 shutdown_connections 2024-04-25T11:38:32.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f925a130640 1 -- 172.21.15.40:0/1364971720 wait complete. 2024-04-25T11:38:32.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7f5ee4074bc0 msgr2=0x7f5ee4076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.317+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 --> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d74002d50 con 0x7f4d70074da0 2024-04-25T11:38:32.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f5f0635d640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7f5ee4074bc0 0x7f5ee4076fe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f4d7effd640 1 -- 172.21.15.40:0/3338584048 <== osd.3 v2:172.21.15.40:6818/364049698 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4d74002d50 con 0x7f4d70074da0 2024-04-25T11:38:32.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 --> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0fc002dd0 con 0x7fa108074c40 2024-04-25T11:38:32.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7fa103fff640 1 -- 172.21.15.40:0/2001097725 <== osd.5 v2:172.21.15.62:6824/3210426647 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa0fc002dd0 con 0x7fa108074c40 2024-04-25T11:38:32.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5ee406c940 msgr2=0x7f5ee406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7f4d70074da0 msgr2=0x7f4d700771c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f4d8f59e640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7f4d70074da0 0x7f4d700771c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f5f0635d640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5ee406c940 0x7f5ee406ee00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5efc037000 tx=0x7f5efc00a040 comp rx=0 tx=0).stop 2024-04-25T11:38:32.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ef80d5200 msgr2=0x7f5ef80de600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4d7006cb20 msgr2=0x7f4d7006efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f4d8f59e640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f4d7006cb20 0x7f4d7006efe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d88033030 tx=0x7f4d88039000 comp rx=0 tx=0).stop 2024-04-25T11:38:32.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f5f0635d640 1 --2- 172.21.15.40:0/4003648555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ef80d5200 0x7f5ef80de600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000b490 tx=0x7f5ef000b950 comp rx=0 tx=0).stop 2024-04-25T11:38:32.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d9006bfc0 msgr2=0x7f4d901f0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f4d8f59e640 1 --2- 172.21.15.40:0/3338584048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d9006bfc0 0x7f4d901f0490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400b3f0 tx=0x7f4d8400b8b0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f5f05b5c640 1 -- 172.21.15.40:0/4003648555 reap_dead start 2024-04-25T11:38:32.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 shutdown_connections 2024-04-25T11:38:32.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.318+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 >> 172.21.15.40:0/4003648555 conn(0x7f5ef8050180 msgr2=0x7f5ef8050560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f3f68ff9640 1 -- 172.21.15.40:0/2149636270 --> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f7c150ae0 con 0x7f3f5c074d60 2024-04-25T11:38:32.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f4d8ed9d640 1 -- 172.21.15.40:0/3338584048 reap_dead start 2024-04-25T11:38:32.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 shutdown_connections 2024-04-25T11:38:32.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 >> 172.21.15.40:0/3338584048 conn(0x7f4d900cc0d0 msgr2=0x7f4d900c2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 shutdown_connections 2024-04-25T11:38:32.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7fa108074c40 msgr2=0x7fa108077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7fa1220e1640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7fa108074c40 0x7fa108077060 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f5f0635d640 1 -- 172.21.15.40:0/4003648555 wait complete. 2024-04-25T11:38:32.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f3f6affd640 1 -- 172.21.15.40:0/2149636270 <== osd.4 v2:172.21.15.62:6800/2239863089 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3f7c150ae0 con 0x7f3f5c074d60 2024-04-25T11:38:32.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 shutdown_connections 2024-04-25T11:38:32.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f4d8f59e640 1 -- 172.21.15.40:0/3338584048 wait complete. 2024-04-25T11:38:32.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f3f68ff9640 1 -- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f3f5c074d60 msgr2=0x7f3f5c077180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa10806c9c0 msgr2=0x7fa10806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7fa1220e1640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa10806c9c0 0x7fa10806ee80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa114033000 tx=0x7fa114009000 comp rx=0 tx=0).stop 2024-04-25T11:38:32.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.319+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 --> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d6c1f5990 con 0x7f0d58074b90 2024-04-25T11:38:32.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7f3f68ff9640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f3f5c074d60 0x7f3f5c077180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 msgr2=0x7fa11c149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7fa1220e1640 1 --2- 172.21.15.40:0/2001097725 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa11c0cb120 0x7fa11c149370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00e9e0 tx=0x7fa10c00eea0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7fa11bfff640 1 -- 172.21.15.40:0/2001097725 reap_dead start 2024-04-25T11:38:32.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7f3f68ff9640 1 -- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3f5c06cb60 msgr2=0x7f3f5c06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7f3f68ff9640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3f5c06cb60 0x7f3f5c06f020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c14d920 tx=0x7f3f70009190 comp rx=0 tx=0).stop 2024-04-25T11:38:32.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 shutdown_connections 2024-04-25T11:38:32.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 >> 172.21.15.40:0/2001097725 conn(0x7fa11c073be0 msgr2=0x7fa11c0ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7f3f68ff9640 1 -- 172.21.15.40:0/2149636270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f7c072710 msgr2=0x7f3f7c1e3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7f3f68ff9640 1 --2- 172.21.15.40:0/2149636270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f7c072710 0x7f3f7c1e3610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00ea10 tx=0x7f3f6c00eed0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7f3f7ad9d640 1 -- 172.21.15.40:0/2149636270 reap_dead start 2024-04-25T11:38:32.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 --> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fae640cca50 con 0x7fae44074c40 2024-04-25T11:38:32.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7f3f68ff9640 1 -- 172.21.15.40:0/2149636270 shutdown_connections 2024-04-25T11:38:32.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.320+0000 7f3f68ff9640 1 -- 172.21.15.40:0/2149636270 >> 172.21.15.40:0/2149636270 conn(0x7f3f7c06d2e0 msgr2=0x7f3f7c14c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f3f68ff9640 1 -- 172.21.15.40:0/2149636270 shutdown_connections 2024-04-25T11:38:32.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 shutdown_connections 2024-04-25T11:38:32.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f0d697fa640 1 -- 172.21.15.40:0/3711069905 <== osd.0 v2:172.21.15.40:6816/2617970798 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0d6c1f5990 con 0x7f0d58074b90 2024-04-25T11:38:32.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7fae5bfff640 1 -- 172.21.15.40:0/1426402920 <== osd.7 v2:172.21.15.62:6816/3160032582 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fae640cca50 con 0x7fae44074c40 2024-04-25T11:38:32.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7fa1220e1640 1 -- 172.21.15.40:0/2001097725 wait complete. 2024-04-25T11:38:32.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f3f68ff9640 1 -- 172.21.15.40:0/2149636270 wait complete. 2024-04-25T11:38:32.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f0d58074b90 msgr2=0x7f0d58076fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f0d7323c640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f0d58074b90 0x7f0d58076fb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d5806c970 msgr2=0x7f0d5806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f0d7323c640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0d5806c970 0x7f0d5806ee30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0d64033000 tx=0x7f0d64009000 comp rx=0 tx=0).stop 2024-04-25T11:38:32.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d300 msgr2=0x7f0d6c1f98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f0d7323c640 1 --2- 172.21.15.40:0/3711069905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d6c06d300 0x7f0d6c1f98b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00b480 tx=0x7f0d5c00b940 comp rx=0 tx=0).stop 2024-04-25T11:38:32.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.321+0000 7f0d717d9640 1 -- 172.21.15.40:0/3711069905 reap_dead start 2024-04-25T11:38:32.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.322+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 shutdown_connections 2024-04-25T11:38:32.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.322+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 >> 172.21.15.40:0/3711069905 conn(0x7f0d6c0713c0 msgr2=0x7f0d6c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.322+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7fae44074c40 msgr2=0x7fae44077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.322+0000 7fae69a43640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7fae44074c40 0x7fae44077060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.322+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 shutdown_connections 2024-04-25T11:38:32.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.322+0000 7f0d7323c640 1 -- 172.21.15.40:0/3711069905 wait complete. 2024-04-25T11:38:32.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.323+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae4406c9c0 msgr2=0x7fae4406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.323+0000 7fae69a43640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fae4406c9c0 0x7fae4406ee80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae54007f70 tx=0x7fae54007f00 comp rx=0 tx=0).stop 2024-04-25T11:38:32.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.323+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae64070f70 msgr2=0x7fae6415cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.323+0000 7fae69a43640 1 --2- 172.21.15.40:0/1426402920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae64070f70 0x7fae6415cc20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae4c000c00 tx=0x7fae4c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.323+0000 7fae63fff640 1 -- 172.21.15.40:0/1426402920 reap_dead start 2024-04-25T11:38:32.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.323+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 shutdown_connections 2024-04-25T11:38:32.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.323+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 >> 172.21.15.40:0/1426402920 conn(0x7fae64067890 msgr2=0x7fae641523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.324+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 shutdown_connections 2024-04-25T11:38:32.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.324+0000 7fae69a43640 1 -- 172.21.15.40:0/1426402920 wait complete. 2024-04-25T11:38:32.365 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:32.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:38:32.366 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:32.367 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T11:38:32.367 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:32.368 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:38:32.368 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:32.369 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:38:32.369 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:32.370 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:38:32.370 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:32.371 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T11:38:32.371 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:32.371 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T11:38:32.372 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:32.373 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T11:38:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.593+0000 7f03f6f20640 1 Processor -- start 2024-04-25T11:38:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.593+0000 7f92362fa640 1 Processor -- start 2024-04-25T11:38:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.593+0000 7f92362fa640 1 -- start start 2024-04-25T11:38:32.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.592+0000 7f48da489640 1 Processor -- start 2024-04-25T11:38:32.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.593+0000 7f48da489640 1 -- start start 2024-04-25T11:38:32.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f6e54822640 1 Processor -- start 2024-04-25T11:38:32.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f6e54822640 1 -- start start 2024-04-25T11:38:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f48da489640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 0x7f48d40727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f48da489640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48d40707e0 0x7f48d4070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f48da489640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d40c4d50 con 0x7f48d4151e40 2024-04-25T11:38:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f48da489640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f48d4073bc0 con 0x7f48d40707e0 2024-04-25T11:38:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f48da489640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f48d40c4a70 con 0x7f48d4151a60 2024-04-25T11:38:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f48d37fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f48d4151e40 0x7f48d41583c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43950/0 (socket says 172.21.15.40:43950) 2024-04-25T11:38:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f48d37fe640 1 -- 172.21.15.40:0/3209676570 learned_addr learned my addr 172.21.15.40:0/3209676570 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f48d8a26640 1 --2- 172.21.15.40:0/3209676570 >> v2:172.21.15.40:3300/0 conn(0x7f48d40707e0 0x7f48d4070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.594+0000 7f48d3fff640 1 --2- 172.21.15.40:0/3209676570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 0x7f48d40727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f92362fa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92300c8170 0x7f92300732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f92362fa640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300c74e0 0x7f92300cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e54822640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6e480d9090 0x7f6e480deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e54822640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48050180 0x7f6e480505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e54822640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e48037160 0x7f6e480375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e54822640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6e48038090 con 0x7f6e48037160 2024-04-25T11:38:32.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e54822640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f6e480df4a0 con 0x7f6e480d9090 2024-04-25T11:38:32.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e54822640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6e480df780 con 0x7f6e48050180 2024-04-25T11:38:32.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.593+0000 7f03f6f20640 1 -- start start 2024-04-25T11:38:32.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f03f6f20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03f01590c0 0x7f03f015f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f03f4cbc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03f01590c0 0x7f03f015f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f922f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92300c8170 0x7f92300732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f922ffff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f92301592c0 0x7f923015fae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:54290/0 (socket says 172.21.15.40:54290) 2024-04-25T11:38:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f922ffff640 1 -- 172.21.15.40:0/2175416274 learned_addr learned my addr 172.21.15.40:0/2175416274 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f92362fa640 1 -- 172.21.15.40:0/2175416274 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9230071f20 con 0x7f92301592c0 2024-04-25T11:38:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e4ed9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6e480d9090 0x7f6e480deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e4ed9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6e480d9090 0x7f6e480deda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51564/0 (socket says 172.21.15.40:51564) 2024-04-25T11:38:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e4ed9d640 1 -- 172.21.15.40:0/3974487347 learned_addr learned my addr 172.21.15.40:0/3974487347 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f92362fa640 1 -- 172.21.15.40:0/2175416274 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f92300c7dd0 con 0x7f92300c8170 2024-04-25T11:38:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f92362fa640 1 -- 172.21.15.40:0/2175416274 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9230072e10 con 0x7f92300c74e0 2024-04-25T11:38:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e4e59c640 1 --2- 172.21.15.40:0/3974487347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48050180 0x7f6e480505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f6e4f59e640 1 --2- 172.21.15.40:0/3974487347 >> v2:172.21.15.40:3300/0 conn(0x7f6e48037160 0x7f6e480375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f9fad638640 1 Processor -- start 2024-04-25T11:38:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f03f4cbc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03f01590c0 0x7f03f015f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57688/0 (socket says 172.21.15.40:57688) 2024-04-25T11:38:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f9fad638640 1 -- start start 2024-04-25T11:38:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730043113 0 0) 0x7f9230071f20 con 0x7f92301592c0 2024-04-25T11:38:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9218003520 con 0x7f92301592c0 2024-04-25T11:38:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f03f4cbc640 1 -- 172.21.15.40:0/654784433 learned_addr learned my addr 172.21.15.40:0/654784433 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f9775ac2640 1 Processor -- start 2024-04-25T11:38:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f9775ac2640 1 -- start start 2024-04-25T11:38:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7fd69317d640 1 Processor -- start 2024-04-25T11:38:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f03f6f20640 1 --2- 172.21.15.40:0/654784433 >> v2:172.21.15.62:3300/0 conn(0x7f03f0072b70 0x7f03f0072fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f48d8a26640 1 -- 172.21.15.40:0/3209676570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 msgr2=0x7f48d40727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f48d8a26640 1 --2- 172.21.15.40:0/3209676570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 0x7f48d40727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f48d8a26640 1 -- 172.21.15.40:0/3209676570 >> v1:172.21.15.40:6789/0 conn(0x7f48d4151e40 legacy=0x7f48d41583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.595+0000 7f48d8a26640 1 -- 172.21.15.40:0/3209676570 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d406fa20 con 0x7f48d40707e0 2024-04-25T11:38:32.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7fabaaad4640 1 Processor -- start 2024-04-25T11:38:32.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f03f54bd640 1 --2- 172.21.15.40:0/654784433 >> v2:172.21.15.62:3300/0 conn(0x7f03f0072b70 0x7f03f0072fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f48d8a26640 1 --2- 172.21.15.40:0/3209676570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d40707e0 0x7f48d4070c50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009d90 tx=0x7f48cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ebdcd3d231be3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f03f6f20640 1 -- 172.21.15.40:0/654784433 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f0073520 con 0x7f03f006f950 2024-04-25T11:38:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f6e4e59c640 1 -- 172.21.15.40:0/3974487347 >> v2:172.21.15.62:3300/0 conn(0x7f6e480d9090 msgr2=0x7f6e480deda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f6e4e59c640 1 --2- 172.21.15.40:0/3974487347 >> v2:172.21.15.62:3300/0 conn(0x7f6e480d9090 0x7f6e480deda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f6e4e59c640 1 -- 172.21.15.40:0/3974487347 >> v2:172.21.15.40:3300/0 conn(0x7f6e48037160 msgr2=0x7f6e480375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f6e4e59c640 1 --2- 172.21.15.40:0/3974487347 >> v2:172.21.15.40:3300/0 conn(0x7f6e48037160 0x7f6e480375d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f6e4e59c640 1 -- 172.21.15.40:0/3974487347 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e480dfa60 con 0x7f6e48050180 2024-04-25T11:38:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f03f6f20640 1 -- 172.21.15.40:0/654784433 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f03f00726e0 con 0x7f03f01590c0 2024-04-25T11:38:32.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f03f6f20640 1 -- 172.21.15.40:0/654784433 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f03f00c82d0 con 0x7f03f0072b70 2024-04-25T11:38:32.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f6e4e59c640 1 --2- 172.21.15.40:0/3974487347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48050180 0x7f6e480505c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e5002db50 tx=0x7f6e500a62c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8bca5df6f6c1bca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f48d2ffd640 1 -- 172.21.15.40:0/3209676570 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc007740 con 0x7f48d40707e0 2024-04-25T11:38:32.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f48d2ffd640 1 -- 172.21.15.40:0/3209676570 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48cc0078e0 con 0x7f48d40707e0 2024-04-25T11:38:32.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f48d2ffd640 1 -- 172.21.15.40:0/3209676570 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc007cb0 con 0x7f48d40707e0 2024-04-25T11:38:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f03f4cbc640 1 -- 172.21.15.40:0/654784433 >> v2:172.21.15.62:3300/0 conn(0x7f03f0072b70 msgr2=0x7f03f0072fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f03f4cbc640 1 --2- 172.21.15.40:0/654784433 >> v2:172.21.15.62:3300/0 conn(0x7f03f0072b70 0x7f03f0072fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f03f4cbc640 1 -- 172.21.15.40:0/654784433 >> v1:172.21.15.62:6789/0 conn(0x7f03f006f950 legacy=0x7f03f006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f9fad638640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9fa8151020 0x7f9fa80cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f9fa7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9fa8151020 0x7f9fa80cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fa7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9fa8151020 0x7f9fa80cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57716/0 (socket says 172.21.15.40:57716) 2024-04-25T11:38:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fa7fff640 1 -- 172.21.15.40:0/955149099 learned_addr learned my addr 172.21.15.40:0/955149099 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fad638640 1 --2- 172.21.15.40:0/955149099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa80c8bd0 0x7f9fa80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fad638640 1 --2- 172.21.15.40:0/955149099 >> v2:172.21.15.62:3300/0 conn(0x7f9fa8072440 0x7f9fa8157130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fad638640 1 -- 172.21.15.40:0/955149099 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa80c9020 con 0x7f9fa8151020 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fad638640 1 -- 172.21.15.40:0/955149099 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa80c8000 con 0x7f9fa8072440 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fad638640 1 -- 172.21.15.40:0/955149099 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa80c9270 con 0x7f9fa80c8bd0 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fa77fe640 1 --2- 172.21.15.40:0/955149099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa80c8bd0 0x7f9fa80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9face37640 1 --2- 172.21.15.40:0/955149099 >> v2:172.21.15.62:3300/0 conn(0x7f9fa8072440 0x7f9fa8157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f9234897640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300c74e0 0x7f92300cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4223422913 0 0) 0x7f9218003520 con 0x7f92301592c0 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300c74e0 msgr2=0x7f92300cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f922effd640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300c74e0 0x7f92300cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 >> v2:172.21.15.40:3300/0 conn(0x7f92300c8170 msgr2=0x7f92300732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f922effd640 1 --2- 172.21.15.40:0/2175416274 >> v2:172.21.15.40:3300/0 conn(0x7f92300c8170 0x7f92300732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9230072200 con 0x7f92301592c0 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7f9224001690 con 0x7f92301592c0 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 >> v1:172.21.15.62:6789/0 conn(0x7f92301592c0 legacy=0x7f923015fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7f922effd640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92180039b0 0x7f9218003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922ffff640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92180039b0 0x7f9218003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922effd640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9218004800 0x7f9218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922effd640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300c74e0 0x7f92180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f92300c7dd0 con 0x7f92180039b0 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9230072e10 con 0x7f9218004800 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f921800bad0 con 0x7f92300c74e0 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922ffff640 1 -- 172.21.15.40:0/2175416274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300c74e0 msgr2=0x7f92180051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922ffff640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300c74e0 0x7f92180051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922ffff640 1 -- 172.21.15.40:0/2175416274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9218004800 msgr2=0x7f9218004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922ffff640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9218004800 0x7f9218004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f922ffff640 1 -- 172.21.15.40:0/2175416274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9218003520 con 0x7f92180039b0 2024-04-25T11:38:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f922f7fe640 1 --2- 172.21.15.40:0/2175416274 >> v2:172.21.15.40:3300/0 conn(0x7f92300c8170 0x7f92300732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7fd69317d640 1 -- start start 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7fd69317d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd68c072560 0x7fd68c15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7fd69317d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd68c157870 0x7fd68c161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7fd69317d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd68c156db0 con 0x7fd68c14d8f0 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7fd69317d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd68c0c42b0 con 0x7fd68c157870 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7fd69317d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd68c0c7940 con 0x7fd68c072560 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7fd69171a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd68c157870 0x7fd68c161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fd69171a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd68c157870 0x7fd68c161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57718/0 (socket says 172.21.15.40:57718) 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fd69171a640 1 -- 172.21.15.40:0/1830890424 learned_addr learned my addr 172.21.15.40:0/1830890424 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fd68bfff640 1 --2- 172.21.15.40:0/1830890424 >> v2:172.21.15.62:3300/0 conn(0x7fd68c072560 0x7fd68c15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f48da489640 1 -- 172.21.15.40:0/3209676570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d40707e0 msgr2=0x7f48d4070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f48da489640 1 --2- 172.21.15.40:0/3209676570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d40707e0 0x7f48d4070c50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009d90 tx=0x7f48cc02a660 comp rx=0 tx=0).stop 2024-04-25T11:38:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f48da489640 1 -- 172.21.15.40:0/3209676570 shutdown_connections 2024-04-25T11:38:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f48da489640 1 --2- 172.21.15.40:0/3209676570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d40707e0 0x7f48d4070c50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f48da489640 1 --2- 172.21.15.40:0/3209676570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 0x7f48d40727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f48da489640 1 -- 172.21.15.40:0/3209676570 >> 172.21.15.40:0/3209676570 conn(0x7f48d40cd170 msgr2=0x7f48d40cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f6e4dd9b640 1 -- 172.21.15.40:0/3974487347 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e500b3070 con 0x7f6e48050180 2024-04-25T11:38:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f6e4dd9b640 1 -- 172.21.15.40:0/3974487347 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e500ae640 con 0x7f6e48050180 2024-04-25T11:38:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f6e54822640 1 -- 172.21.15.40:0/3974487347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48050180 msgr2=0x7f6e480505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f6e54822640 1 --2- 172.21.15.40:0/3974487347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48050180 0x7f6e480505c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e5002db50 tx=0x7f6e500a62c0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f9775ac2640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152660 0x7f977015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f9775ac2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97700c83c0 0x7f9770163150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f9775ac2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97700ccbf0 con 0x7f9770152a40 2024-04-25T11:38:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f9775ac2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f97700cc9a0 con 0x7f97700c83c0 2024-04-25T11:38:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f9775ac2640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f97700cced0 con 0x7f9770152660 2024-04-25T11:38:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f976f7fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152660 0x7f977015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f976f7fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152660 0x7f977015bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40838/0 (socket says 172.21.15.40:40838) 2024-04-25T11:38:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f976f7fe640 1 -- 172.21.15.40:0/86526048 learned_addr learned my addr 172.21.15.40:0/86526048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.596+0000 7fabaaad4640 1 -- start start 2024-04-25T11:38:32.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fabaaad4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faba406e3a0 0x7faba415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fabaaad4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba40c7360 con 0x7faba4158f80 2024-04-25T11:38:32.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fabaaad4640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba40c6da0 con 0x7faba4073a60 2024-04-25T11:38:32.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fabaaad4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7faba415fd00 con 0x7faba406e3a0 2024-04-25T11:38:32.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7faba92d1640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7faba4073a60 0x7faba406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:54318/0 (socket says 172.21.15.40:54318) 2024-04-25T11:38:32.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7faba92d1640 1 -- 172.21.15.40:0/3161785529 learned_addr learned my addr 172.21.15.40:0/3161785529 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:32.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f976ffff640 1 --2- 172.21.15.40:0/86526048 >> v2:172.21.15.40:3300/0 conn(0x7f97700c83c0 0x7f9770163150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fabaa2d3640 1 --2- 172.21.15.40:0/3161785529 >> v2:172.21.15.40:3300/0 conn(0x7faba406e3a0 0x7faba415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f6e54822640 1 -- 172.21.15.40:0/3974487347 shutdown_connections 2024-04-25T11:38:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f6e54822640 1 --2- 172.21.15.40:0/3974487347 >> v2:172.21.15.40:3300/0 conn(0x7f6e48037160 0x7f6e480375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f6e54822640 1 --2- 172.21.15.40:0/3974487347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48050180 0x7f6e480505c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f6e54822640 1 --2- 172.21.15.40:0/3974487347 >> v2:172.21.15.62:3300/0 conn(0x7f6e480d9090 0x7f6e480deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f6e54822640 1 -- 172.21.15.40:0/3974487347 >> 172.21.15.40:0/3974487347 conn(0x7f6e48054610 msgr2=0x7f6e48054a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fa77fe640 1 -- 172.21.15.40:0/955149099 >> v2:172.21.15.62:3300/0 conn(0x7f9fa8072440 msgr2=0x7f9fa8157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fa77fe640 1 --2- 172.21.15.40:0/955149099 >> v2:172.21.15.62:3300/0 conn(0x7f9fa8072440 0x7f9fa8157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fa77fe640 1 -- 172.21.15.40:0/955149099 >> v2:172.21.15.40:3300/0 conn(0x7f9fa8151020 msgr2=0x7f9fa80cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fa77fe640 1 --2- 172.21.15.40:0/955149099 >> v2:172.21.15.40:3300/0 conn(0x7f9fa8151020 0x7f9fa80cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f9fa77fe640 1 -- 172.21.15.40:0/955149099 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa80c9550 con 0x7f9fa80c8bd0 2024-04-25T11:38:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7f48da489640 1 -- 172.21.15.40:0/3209676570 shutdown_connections 2024-04-25T11:38:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f6e54822640 1 -- 172.21.15.40:0/3974487347 shutdown_connections 2024-04-25T11:38:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f9fa7fff640 1 --2- 172.21.15.40:0/955149099 >> v2:172.21.15.40:3300/0 conn(0x7f9fa8151020 0x7f9fa80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388265858 0 0) 0x7fd68c156db0 con 0x7fd68c14d8f0 2024-04-25T11:38:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.598+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd678003520 con 0x7fd68c14d8f0 2024-04-25T11:38:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 747808842 0 0) 0x7fd678003520 con 0x7fd68c14d8f0 2024-04-25T11:38:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 >> v2:172.21.15.62:3300/0 conn(0x7fd68c072560 msgr2=0x7fd68c15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 --2- 172.21.15.40:0/1830890424 >> v2:172.21.15.62:3300/0 conn(0x7fd68c072560 0x7fd68c15a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 >> v2:172.21.15.40:3300/0 conn(0x7fd68c157870 msgr2=0x7fd68c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7faba8ad0640 1 -- 172.21.15.40:0/3161785529 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4144319524 0 0) 0x7faba40c6da0 con 0x7faba4073a60 2024-04-25T11:38:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 --2- 172.21.15.40:0/1830890424 >> v2:172.21.15.40:3300/0 conn(0x7fd68c157870 0x7fd68c161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c0c7c20 con 0x7fd68c14d8f0 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7fd6840019f0 con 0x7fd68c14d8f0 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 >> v1:172.21.15.62:6789/0 conn(0x7fd68c14d8f0 legacy=0x7fd68c0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6780039b0 0x7fd678003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd678004800 0x7fd678004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd67800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c0c7940 con 0x7fd678004800 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd67800bf00 con 0x7fd6780039b0 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd67800c0d0 con 0x7fd6780051c0 2024-04-25T11:38:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7faba8ad0640 1 -- 172.21.15.40:0/3161785529 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab90003520 con 0x7faba4073a60 2024-04-25T11:38:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.597+0000 7f03f4cbc640 1 -- 172.21.15.40:0/654784433 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f00c8570 con 0x7f03f01590c0 2024-04-25T11:38:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f48da489640 1 -- 172.21.15.40:0/3209676570 wait complete. 2024-04-25T11:38:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f976f7fe640 1 -- 172.21.15.40:0/86526048 >> v2:172.21.15.40:3300/0 conn(0x7f97700c83c0 msgr2=0x7f9770163150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f922ffff640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92180039b0 0x7f9218003dd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f92300c7990 tx=0x7f9218005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30616fbdd896443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f976f7fe640 1 --2- 172.21.15.40:0/86526048 >> v2:172.21.15.40:3300/0 conn(0x7f97700c83c0 0x7f9770163150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f976f7fe640 1 -- 172.21.15.40:0/86526048 >> v1:172.21.15.40:6789/0 conn(0x7f9770152a40 legacy=0x7f97700c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f976f7fe640 1 -- 172.21.15.40:0/86526048 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9770158970 con 0x7f9770152660 2024-04-25T11:38:32.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd69171a640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd67800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f6e54822640 1 -- 172.21.15.40:0/3974487347 wait complete. 2024-04-25T11:38:32.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7f03f4cbc640 1 --2- 172.21.15.40:0/654784433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f01590c0 0x7f03f015f960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f03f0073d60 tx=0x7f03e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b04748296ea1ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fd690f19640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6780039b0 0x7fd678003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fabaa2d3640 1 -- 172.21.15.40:0/3161785529 >> v1:172.21.15.62:6789/0 conn(0x7faba4073a60 legacy=0x7faba406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7fd68bfff640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd678004800 0x7fd678004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fabaa2d3640 1 -- 172.21.15.40:0/3161785529 >> v1:172.21.15.40:6789/0 conn(0x7faba4158f80 legacy=0x7faba406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.599+0000 7fabaa2d3640 1 -- 172.21.15.40:0/3161785529 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba40c7080 con 0x7faba406e3a0 2024-04-25T11:38:32.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f976f7fe640 1 --2- 172.21.15.40:0/86526048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152660 0x7f977015bdc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9760009870 tx=0x7f976002a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=317f67d880f88427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f03ef7fe640 1 -- 172.21.15.40:0/654784433 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4037070 con 0x7f03f01590c0 2024-04-25T11:38:32.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224007ce0 con 0x7f92180039b0 2024-04-25T11:38:32.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9224034070 con 0x7f92180039b0 2024-04-25T11:38:32.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f922effd640 1 -- 172.21.15.40:0/2175416274 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224005d30 con 0x7f92180039b0 2024-04-25T11:38:32.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7fd69171a640 1 -- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6780039b0 msgr2=0x7fd678003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f03ef7fe640 1 -- 172.21.15.40:0/654784433 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e4032440 con 0x7f03f01590c0 2024-04-25T11:38:32.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f9fa77fe640 1 --2- 172.21.15.40:0/955149099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa80c8bd0 0x7f9fa80cca70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9fa806c860 tx=0x7f9f9802b1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c24700347dc9738 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f9fa6ffd640 1 -- 172.21.15.40:0/955149099 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f980089e0 con 0x7f9fa80c8bd0 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f03ef7fe640 1 -- 172.21.15.40:0/654784433 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4032810 con 0x7f03f01590c0 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f03f6f20640 1 -- 172.21.15.40:0/654784433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f01590c0 msgr2=0x7f03f015f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f03f6f20640 1 --2- 172.21.15.40:0/654784433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f01590c0 0x7f03f015f960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f03f0073d60 tx=0x7f03e402a210 comp rx=0 tx=0).stop 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f03f6f20640 1 -- 172.21.15.40:0/654784433 shutdown_connections 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f03f6f20640 1 --2- 172.21.15.40:0/654784433 >> v2:172.21.15.62:3300/0 conn(0x7f03f0072b70 0x7f03f0072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f03f6f20640 1 --2- 172.21.15.40:0/654784433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f01590c0 0x7f03f015f960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f03f6f20640 1 -- 172.21.15.40:0/654784433 >> 172.21.15.40:0/654784433 conn(0x7f03f00cd200 msgr2=0x7f03f00cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f9fa6ffd640 1 -- 172.21.15.40:0/955149099 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f98008b80 con 0x7f9fa80c8bd0 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7f9fa6ffd640 1 -- 172.21.15.40:0/955149099 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800a0d0 con 0x7f9fa80c8bd0 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f9fad638640 1 -- 172.21.15.40:0/955149099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa80c8bd0 msgr2=0x7f9fa80cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f9fad638640 1 --2- 172.21.15.40:0/955149099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa80c8bd0 0x7f9fa80cca70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9fa806c860 tx=0x7f9f9802b1c0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f9fad638640 1 -- 172.21.15.40:0/955149099 shutdown_connections 2024-04-25T11:38:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f9fad638640 1 --2- 172.21.15.40:0/955149099 >> v2:172.21.15.62:3300/0 conn(0x7f9fa8072440 0x7f9fa8157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f9fad638640 1 --2- 172.21.15.40:0/955149099 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa80c8bd0 0x7f9fa80cca70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f9fad638640 1 --2- 172.21.15.40:0/955149099 >> v2:172.21.15.40:3300/0 conn(0x7f9fa8151020 0x7f9fa80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f92362fa640 1 -- 172.21.15.40:0/2175416274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92180039b0 msgr2=0x7f9218003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f92362fa640 1 --2- 172.21.15.40:0/2175416274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92180039b0 0x7f9218003dd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f92300c7990 tx=0x7f9218005830 comp rx=0 tx=0).stop 2024-04-25T11:38:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f9234897640 1 -- 172.21.15.40:0/2175416274 reap_dead start 2024-04-25T11:38:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f92362fa640 1 -- 172.21.15.40:0/2175416274 shutdown_connections 2024-04-25T11:38:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f92362fa640 1 -- 172.21.15.40:0/2175416274 >> 172.21.15.40:0/2175416274 conn(0x7f923006f740 msgr2=0x7f923006fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7fd69171a640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6780039b0 0x7fd678003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7fd69171a640 1 -- 172.21.15.40:0/1830890424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd678004800 msgr2=0x7fd678004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7fd69171a640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd678004800 0x7fd678004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.600+0000 7fd69171a640 1 -- 172.21.15.40:0/1830890424 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c0c7c20 con 0x7fd6780051c0 2024-04-25T11:38:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7fd69171a640 1 -- 172.21.15.40:0/1830890424 reap_dead start 2024-04-25T11:38:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f48da489640 1 Processor -- start 2024-04-25T11:38:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f48da489640 1 -- start start 2024-04-25T11:38:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f976e7fc640 1 -- 172.21.15.40:0/86526048 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760037070 con 0x7f9770152660 2024-04-25T11:38:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f976e7fc640 1 -- 172.21.15.40:0/86526048 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9760032440 con 0x7f9770152660 2024-04-25T11:38:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f976e7fc640 1 -- 172.21.15.40:0/86526048 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760032810 con 0x7f9770152660 2024-04-25T11:38:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7fabaa2d3640 1 --2- 172.21.15.40:0/3161785529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 0x7faba415f7c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009d90 tx=0x7fab9c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd527016e932fb09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7faba8ad0640 1 -- 172.21.15.40:0/3161785529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c035020 con 0x7faba406e3a0 2024-04-25T11:38:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7faba8ad0640 1 -- 172.21.15.40:0/3161785529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9c007540 con 0x7faba406e3a0 2024-04-25T11:38:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7faba8ad0640 1 -- 172.21.15.40:0/3161785529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94001670 con 0x7faba406e3a0 2024-04-25T11:38:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f03f6f20640 1 -- 172.21.15.40:0/654784433 shutdown_connections 2024-04-25T11:38:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f9fad638640 1 -- 172.21.15.40:0/955149099 >> 172.21.15.40:0/955149099 conn(0x7f9fa806b970 msgr2=0x7f9fa806bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7fd690f19640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6780039b0 0x7fd678003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7fd68bfff640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd678004800 0x7fd678004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f92362fa640 1 -- 172.21.15.40:0/2175416274 shutdown_connections 2024-04-25T11:38:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f6e54822640 1 Processor -- start 2024-04-25T11:38:32.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7f9fad638640 1 -- 172.21.15.40:0/955149099 shutdown_connections 2024-04-25T11:38:32.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7fd69171a640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd67800b9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd67400abd0 tx=0x7fd678005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47dc549384f93d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7f9fad638640 1 -- 172.21.15.40:0/955149099 wait complete. 2024-04-25T11:38:32.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f92362fa640 1 -- 172.21.15.40:0/2175416274 wait complete. 2024-04-25T11:38:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7fabaaad4640 1 -- 172.21.15.40:0/3161785529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 msgr2=0x7faba415f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7fabaaad4640 1 --2- 172.21.15.40:0/3161785529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 0x7faba415f7c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009d90 tx=0x7fab9c02a610 comp rx=0 tx=0).stop 2024-04-25T11:38:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7f48da489640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d40707e0 0x7f48d4155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7f9775ac2640 1 -- 172.21.15.40:0/86526048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152660 msgr2=0x7f977015bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7f6e54822640 1 -- start start 2024-04-25T11:38:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f48da489640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 0x7f48d413d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f9775ac2640 1 --2- 172.21.15.40:0/86526048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152660 0x7f977015bdc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9760009870 tx=0x7f976002a1d0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fabaaad4640 1 -- 172.21.15.40:0/3161785529 shutdown_connections 2024-04-25T11:38:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fabaaad4640 1 --2- 172.21.15.40:0/3161785529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 0x7faba415f7c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fabaaad4640 1 -- 172.21.15.40:0/3161785529 >> 172.21.15.40:0/3161785529 conn(0x7faba40cc0d0 msgr2=0x7faba40cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674038030 con 0x7fd6780051c0 2024-04-25T11:38:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd674004020 con 0x7fd6780051c0 2024-04-25T11:38:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.602+0000 7fd68b7fe640 1 -- 172.21.15.40:0/1830890424 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c009690 con 0x7fd6780051c0 2024-04-25T11:38:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f9775ac2640 1 -- 172.21.15.40:0/86526048 shutdown_connections 2024-04-25T11:38:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f9775ac2640 1 --2- 172.21.15.40:0/86526048 >> v2:172.21.15.40:3300/0 conn(0x7f97700c83c0 0x7f9770163150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f9775ac2640 1 --2- 172.21.15.40:0/86526048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152660 0x7f977015bdc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f9775ac2640 1 -- 172.21.15.40:0/86526048 >> 172.21.15.40:0/86526048 conn(0x7f977006d2e0 msgr2=0x7f977006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f48d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d40707e0 0x7f48d4155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f48d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d40707e0 0x7f48d4155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51610/0 (socket says 172.21.15.40:51610) 2024-04-25T11:38:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f48da489640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d4151e40 0x7f48d41511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fd69317d640 1 -- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 msgr2=0x7fd67800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fd69317d640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd67800b9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd67400abd0 tx=0x7fd678005830 comp rx=0 tx=0).stop 2024-04-25T11:38:32.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fabaaad4640 1 -- 172.21.15.40:0/3161785529 shutdown_connections 2024-04-25T11:38:32.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f92362fa640 1 Processor -- start 2024-04-25T11:38:32.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.601+0000 7f03f6f20640 1 -- 172.21.15.40:0/654784433 wait complete. 2024-04-25T11:38:32.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f03f6f20640 1 Processor -- start 2024-04-25T11:38:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f48d3fff640 1 -- 172.21.15.40:0/1132715742 learned_addr learned my addr 172.21.15.40:0/1132715742 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f48d40702c0 con 0x7f48d4151e40 2024-04-25T11:38:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48d406ffe0 con 0x7f48d40707e0 2024-04-25T11:38:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f48d40c4670 con 0x7f48d4151a60 2024-04-25T11:38:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fabaaad4640 1 -- 172.21.15.40:0/3161785529 wait complete. 2024-04-25T11:38:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f03f6f20640 1 -- start start 2024-04-25T11:38:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f9775ac2640 1 -- 172.21.15.40:0/86526048 shutdown_connections 2024-04-25T11:38:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fd69317d640 1 -- 172.21.15.40:0/1830890424 shutdown_connections 2024-04-25T11:38:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fd69317d640 1 --2- 172.21.15.40:0/1830890424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd67800b9c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7fd69317d640 1 -- 172.21.15.40:0/1830890424 >> 172.21.15.40:0/1830890424 conn(0x7fd68c06f740 msgr2=0x7fd68c06fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f48d37fe640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 0x7f48d413d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f92362fa640 1 -- start start 2024-04-25T11:38:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f6e54822640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48037160 0x7f6e48047ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f9fad638640 1 Processor -- start 2024-04-25T11:38:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f48d8a26640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d4151e40 0x7f48d41511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f48d3fff640 1 -- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 msgr2=0x7f48d413d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f48d3fff640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 0x7f48d413d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f48d3fff640 1 -- 172.21.15.40:0/1132715742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d4151e40 msgr2=0x7f48d41511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f48d3fff640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d4151e40 0x7f48d41511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f48d3fff640 1 -- 172.21.15.40:0/1132715742 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d4138c70 con 0x7f48d40707e0 2024-04-25T11:38:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f48d8a26640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d4151e40 0x7f48d41511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f48d3fff640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d40707e0 0x7f48d4155380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e9b0 tx=0x7f48c400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f9775ac2640 1 -- 172.21.15.40:0/86526048 wait complete. 2024-04-25T11:38:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f6e54822640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e48050180 0x7f6e4804e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f6e54822640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e480d9090 0x7f6e4804e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f6e54822640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6e48038520 con 0x7f6e480d9090 2024-04-25T11:38:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f6e54822640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480d9470 con 0x7f6e48050180 2024-04-25T11:38:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.603+0000 7f6e54822640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6e480386a0 con 0x7f6e48037160 2024-04-25T11:38:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f9fad638640 1 -- start start 2024-04-25T11:38:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f6e4e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e48050180 0x7f6e4804e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f6e4e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e48050180 0x7f6e4804e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51620/0 (socket says 172.21.15.40:51620) 2024-04-25T11:38:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f6e4e59c640 1 -- 172.21.15.40:0/2888741856 learned_addr learned my addr 172.21.15.40:0/2888741856 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7f6e4f59e640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e480d9090 0x7f6e4804e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7fd69317d640 1 -- 172.21.15.40:0/1830890424 shutdown_connections 2024-04-25T11:38:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f92362fa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92180039b0 0x7f923014ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f6e4ed9d640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48037160 0x7f6e48047ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f6f20640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03f006f950 0x7f03f014bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f92362fa640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f923015fdc0 0x7f92300cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f92362fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92300cbaf0 0x7f92300c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f92362fa640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9230072aa0 con 0x7f92300cbaf0 2024-04-25T11:38:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f92362fa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92300727c0 con 0x7f92180039b0 2024-04-25T11:38:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f92362fa640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f92300737e0 con 0x7f923015fdc0 2024-04-25T11:38:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f6f20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f0072b70 0x7f03f01f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f923015fdc0 0x7f92300cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f9234897640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92300cbaf0 0x7f92300c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f6f20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03f01590c0 0x7f03f01464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f923015fdc0 0x7f92300cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40892/0 (socket says 172.21.15.40:40892) 2024-04-25T11:38:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f922f7fe640 1 -- 172.21.15.40:0/622789275 learned_addr learned my addr 172.21.15.40:0/622789275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f922ffff640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92180039b0 0x7f923014ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f6e4f59e640 1 -- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48037160 msgr2=0x7f6e48047ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f6e4f59e640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48037160 0x7f6e48047ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f6e4f59e640 1 -- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e48050180 msgr2=0x7f6e4804e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f6e4f59e640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e48050180 0x7f6e4804e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f6e4f59e640 1 -- 172.21.15.40:0/2888741856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e48048430 con 0x7f6e480d9090 2024-04-25T11:38:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f4cbc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03f006f950 0x7f03f014bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f4cbc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03f006f950 0x7f03f014bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40902/0 (socket says 172.21.15.40:40902) 2024-04-25T11:38:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7fabaaad4640 1 Processor -- start 2024-04-25T11:38:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f4cbc640 1 -- 172.21.15.40:0/1475816348 learned_addr learned my addr 172.21.15.40:0/1475816348 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03f00c8e10 con 0x7f03f0072b70 2024-04-25T11:38:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f54bd640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03f01590c0 0x7f03f01464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f9775ac2640 1 Processor -- start 2024-04-25T11:38:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f03f00c8b30 con 0x7f03f01590c0 2024-04-25T11:38:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f03f00c90f0 con 0x7f03f006f950 2024-04-25T11:38:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9fad638640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa8072440 0x7f9fa80cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9234897640 1 -- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f923015fdc0 msgr2=0x7f92300cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9234897640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f923015fdc0 0x7f92300cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9234897640 1 -- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92180039b0 msgr2=0x7f923014ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9234897640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92180039b0 0x7f923014ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9234897640 1 -- 172.21.15.40:0/622789275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9230162640 con 0x7f92300cbaf0 2024-04-25T11:38:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.604+0000 7fd69317d640 1 -- 172.21.15.40:0/1830890424 wait complete. 2024-04-25T11:38:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9fad638640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa80c8bd0 0x7f9fa8143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9775ac2640 1 -- start start 2024-04-25T11:38:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9fad638640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa8151020 0x7f9fa80ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9fad638640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80c9df0 con 0x7f9fa80c8bd0 2024-04-25T11:38:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9fad638640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80c9b10 con 0x7f9fa8072440 2024-04-25T11:38:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9fad638640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa8156bc0 con 0x7f9fa8151020 2024-04-25T11:38:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f6e4f59e640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e480d9090 0x7f6e4804e610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000d580 tx=0x7f6e4000da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f6e377fe640 1 -- 172.21.15.40:0/2888741856 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40018020 con 0x7f6e480d9090 2024-04-25T11:38:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f03f4cbc640 1 -- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03f01590c0 msgr2=0x7f03f01464c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f03f4cbc640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03f01590c0 0x7f03f01464c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f03f4cbc640 1 -- 172.21.15.40:0/1475816348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f0072b70 msgr2=0x7f03f01f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9face37640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa8151020 0x7f9fa80ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9face37640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa8151020 0x7f9fa80ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:40906/0 (socket says 172.21.15.40:40906) 2024-04-25T11:38:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9face37640 1 -- 172.21.15.40:0/686380992 learned_addr learned my addr 172.21.15.40:0/686380992 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f9234897640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92300cbaf0 0x7f92300c5800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f92300ca1a0 tx=0x7f922000d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f03f4cbc640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f0072b70 0x7f03f01f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f03f4cbc640 1 -- 172.21.15.40:0/1475816348 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f01f9240 con 0x7f03f006f950 2024-04-25T11:38:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7fd69317d640 1 Processor -- start 2024-04-25T11:38:32.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f6e377fe640 1 -- 172.21.15.40:0/2888741856 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e40014620 con 0x7f6e480d9090 2024-04-25T11:38:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7fd69317d640 1 -- start start 2024-04-25T11:38:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4804ccc0 con 0x7f6e480d9090 2024-04-25T11:38:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4804d280 con 0x7f6e480d9090 2024-04-25T11:38:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f922cff9640 1 -- 172.21.15.40:0/622789275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92200125f0 con 0x7f92300cbaf0 2024-04-25T11:38:32.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f03f4cbc640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03f006f950 0x7f03f014bc70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f03e4000c00 tx=0x7f03e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f922cff9640 1 -- 172.21.15.40:0/622789275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9220019070 con 0x7f92300cbaf0 2024-04-25T11:38:32.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7f922cff9640 1 -- 172.21.15.40:0/622789275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92200129d0 con 0x7f92300cbaf0 2024-04-25T11:38:32.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.605+0000 7fabaaad4640 1 -- start start 2024-04-25T11:38:32.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.606+0000 7fabaaad4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 0x7faba4071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f92362fa640 1 -- 172.21.15.40:0/622789275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9230162920 con 0x7f92300cbaf0 2024-04-25T11:38:32.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7fabaaad4640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba4158f80 0x7faba41f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7fabaaad4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba41eaa00 0x7faba40726c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f92362fa640 1 -- 172.21.15.40:0/622789275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923014d9a0 con 0x7f92300cbaf0 2024-04-25T11:38:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f9face37640 1 -- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa8072440 msgr2=0x7f9fa80cbda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:38:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f9face37640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa8072440 0x7f9fa80cbda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f9face37640 1 -- 172.21.15.40:0/686380992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa80c8bd0 msgr2=0x7f9fa8143200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:38:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f9face37640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa80c8bd0 0x7f9fa8143200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f9face37640 1 -- 172.21.15.40:0/686380992 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa81ed060 con 0x7f9fa8151020 2024-04-25T11:38:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7fabaaad4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faba4073ef0 con 0x7faba406e3a0 2024-04-25T11:38:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7fabaaad4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faba406d5a0 con 0x7faba41eaa00 2024-04-25T11:38:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7fabaa2d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba41eaa00 0x7faba40726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7fabaa2d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba41eaa00 0x7faba40726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51650/0 (socket says 172.21.15.40:51650) 2024-04-25T11:38:32.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7fabaa2d3640 1 -- 172.21.15.40:0/2185190765 learned_addr learned my addr 172.21.15.40:0/2185190765 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f9775ac2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97700c83c0 0x7f97701fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faba40c7a00 con 0x7faba4158f80 2024-04-25T11:38:32.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f9775ac2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9770152660 0x7f97700caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f6e377fe640 1 -- 172.21.15.40:0/2888741856 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40014a10 con 0x7f6e480d9090 2024-04-25T11:38:32.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f976f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97700c83c0 0x7f97701fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f48d0ff9640 1 -- 172.21.15.40:0/1132715742 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4017070 con 0x7f48d40707e0 2024-04-25T11:38:32.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f976f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97700c83c0 0x7f97701fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:51660/0 (socket says 172.21.15.40:51660) 2024-04-25T11:38:32.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f48d0ff9640 1 -- 172.21.15.40:0/1132715742 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c400c9a0 con 0x7f48d40707e0 2024-04-25T11:38:32.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d4138f50 con 0x7f48d40707e0 2024-04-25T11:38:32.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f976f7fe640 1 -- 172.21.15.40:0/1286936063 learned_addr learned my addr 172.21.15.40:0/1286936063 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f48d0ff9640 1 -- 172.21.15.40:0/1132715742 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c400cd90 con 0x7f48d40707e0 2024-04-25T11:38:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f9face37640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa8151020 0x7f9fa80ca7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00ef00 tx=0x7f9f9c010e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d413d0e0 con 0x7f48d40707e0 2024-04-25T11:38:32.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f9775ac2640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152a40 0x7f97701635d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97700c9860 con 0x7f9770152660 2024-04-25T11:38:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9770158f30 con 0x7f97700c83c0 2024-04-25T11:38:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f97700c99e0 con 0x7f9770152a40 2024-04-25T11:38:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fd69317d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd68c1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fd69317d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd68c148e10 0x7fd68c14ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fd69317d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c14bfa0 0x7fd68c0cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fd69317d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c157c80 con 0x7fd68c148e10 2024-04-25T11:38:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fd69317d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c157350 con 0x7fd68c14bfa0 2024-04-25T11:38:32.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fd69317d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd68c0c3680 con 0x7fd6780051c0 2024-04-25T11:38:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f976effd640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9770152660 0x7f97700caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7faba92d1640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba4158f80 0x7faba41f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f03ed7fa640 1 -- 172.21.15.40:0/1475816348 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4041020 con 0x7f03f006f950 2024-04-25T11:38:32.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.607+0000 7f03ed7fa640 1 -- 172.21.15.40:0/1475816348 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e40306c0 con 0x7f03f006f950 2024-04-25T11:38:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f03ed7fa640 1 -- 172.21.15.40:0/1475816348 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4037040 con 0x7f03f006f950 2024-04-25T11:38:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f00c6cf0 con 0x7f03f006f950 2024-04-25T11:38:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fabaa2d3640 1 -- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba4158f80 msgr2=0x7faba41f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fabaa2d3640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba4158f80 0x7faba41f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fabaa2d3640 1 -- 172.21.15.40:0/2185190765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 msgr2=0x7faba4071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7faba9ad2640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 0x7faba4071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f976ffff640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152a40 0x7f97701635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fd68bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd68c148e10 0x7fd68c14ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f014c600 con 0x7f03f006f950 2024-04-25T11:38:32.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f9fa4ff9640 1 -- 172.21.15.40:0/686380992 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c009d10 con 0x7f9fa8151020 2024-04-25T11:38:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fabaa2d3640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 0x7faba4071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fabaa2d3640 1 -- 172.21.15.40:0/2185190765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba41e8230 con 0x7faba41eaa00 2024-04-25T11:38:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7faba9ad2640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 0x7faba4071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7f92167fc640 1 -- 172.21.15.40:0/622789275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9230155540 con 0x7f92300cbaf0 2024-04-25T11:38:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7fabaa2d3640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba41eaa00 0x7faba40726c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009d90 tx=0x7fab9c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7fab9a7fc640 1 -- 172.21.15.40:0/2185190765 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c03f070 con 0x7faba41eaa00 2024-04-25T11:38:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fd68bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd68c148e10 0x7fd68c14ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57798/0 (socket says 172.21.15.40:57798) 2024-04-25T11:38:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7fd68bfff640 1 -- 172.21.15.40:0/97903959 learned_addr learned my addr 172.21.15.40:0/97903959 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7fd690f19640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd68c1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7fab9a7fc640 1 -- 172.21.15.40:0/2185190765 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9c003b90 con 0x7faba41eaa00 2024-04-25T11:38:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7fab9a7fc640 1 -- 172.21.15.40:0/2185190765 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c035070 con 0x7faba41eaa00 2024-04-25T11:38:32.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7fd69171a640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c14bfa0 0x7fd68c0cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa8143740 con 0x7f9fa8151020 2024-04-25T11:38:32.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa80cb7e0 con 0x7f9fa8151020 2024-04-25T11:38:32.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7f9fa4ff9640 1 -- 172.21.15.40:0/686380992 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9c015860 con 0x7f9fa8151020 2024-04-25T11:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba41e8510 con 0x7faba41eaa00 2024-04-25T11:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.610+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba41e8ad0 con 0x7faba41eaa00 2024-04-25T11:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.609+0000 7f9fa4ff9640 1 -- 172.21.15.40:0/686380992 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c015c50 con 0x7f9fa8151020 2024-04-25T11:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.610+0000 7fd690f19640 1 -- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c14bfa0 msgr2=0x7fd68c0cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.610+0000 7fd690f19640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c14bfa0 0x7fd68c0cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.610+0000 7fd690f19640 1 -- 172.21.15.40:0/97903959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd68c148e10 msgr2=0x7fd68c14ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.610+0000 7fd690f19640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd68c148e10 0x7fd68c14ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.610+0000 7fd690f19640 1 -- 172.21.15.40:0/97903959 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c1ecc70 con 0x7fd6780051c0 2024-04-25T11:38:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f976f7fe640 1 -- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152a40 msgr2=0x7f97701635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f976f7fe640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152a40 0x7f97701635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f976f7fe640 1 -- 172.21.15.40:0/1286936063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9770152660 msgr2=0x7f97700caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f976f7fe640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9770152660 0x7f97700caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.608+0000 7f976f7fe640 1 -- 172.21.15.40:0/1286936063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9770154b40 con 0x7f97700c83c0 2024-04-25T11:38:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.610+0000 7f03ed7fa640 1 -- 172.21.15.40:0/1475816348 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03e4044020 con 0x7f03f006f950 2024-04-25T11:38:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.611+0000 7f03ed7fa640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03d406c9c0 0x7f03d406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.611+0000 7f03effff640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03d406c9c0 0x7f03d406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.611+0000 7f03ed7fa640 1 -- 172.21.15.40:0/1475816348 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f03e40ac250 con 0x7f03f006f950 2024-04-25T11:38:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.611+0000 7f48ba7fc640 1 -- 172.21.15.40:0/1132715742 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d404fd50 con 0x7f48d40707e0 2024-04-25T11:38:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.611+0000 7f48d0ff9640 1 -- 172.21.15.40:0/1132715742 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48c401a070 con 0x7f48d40707e0 2024-04-25T11:38:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f0153110 con 0x7f03f006f950 2024-04-25T11:38:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.611+0000 7f6e377fe640 1 -- 172.21.15.40:0/2888741856 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e4001b020 con 0x7f6e480d9090 2024-04-25T11:38:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.611+0000 7f6e377fe640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6e3c06c940 0x7f6e3c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f6e4ed9d640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6e3c06c940 0x7f6e3c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba40ca470 con 0x7faba41eaa00 2024-04-25T11:38:32.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f03effff640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03d406c9c0 0x7f03d406ee80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03f015ca20 tx=0x7f03dc007950 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f88005290 con 0x7f9fa8151020 2024-04-25T11:38:32.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f976f7fe640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97700c83c0 0x7f97701fcef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f976002a7c0 tx=0x7f97600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7fd690f19640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd68c1597d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd68400ebd0 tx=0x7fd68400c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f48d0ff9640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48c006c9c0 0x7f48c006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7fab9a7fc640 1 -- 172.21.15.40:0/2185190765 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab9c042070 con 0x7faba41eaa00 2024-04-25T11:38:32.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.611+0000 7f922cff9640 1 -- 172.21.15.40:0/622789275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9220012b70 con 0x7f92300cbaf0 2024-04-25T11:38:32.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f922cff9640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f921c06c9f0 0x7f921c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f922cff9640 1 -- 172.21.15.40:0/622789275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f922008c5b0 con 0x7f92300cbaf0 2024-04-25T11:38:32.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e30005290 con 0x7f6e480d9090 2024-04-25T11:38:32.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f48d0ff9640 1 -- 172.21.15.40:0/1132715742 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f48c408bf00 con 0x7f48d40707e0 2024-04-25T11:38:32.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f6e4ed9d640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6e3c06c940 0x7f6e3c06ee00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098e0 tx=0x7f6e44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f48d37fe640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48c006c9c0 0x7f48c006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.612+0000 7f922ffff640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f921c06c9f0 0x7f921c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.613+0000 7fab9a7fc640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fab8806c8f0 0x7fab8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.613+0000 7fab9a7fc640 1 -- 172.21.15.40:0/2185190765 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fab9c0acbc0 con 0x7faba41eaa00 2024-04-25T11:38:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.613+0000 7faba9ad2640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fab8806c8f0 0x7fab8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.613+0000 7f48d37fe640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48c006c9c0 0x7f48c006ee80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48d40cc1d0 tx=0x7f48c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.613+0000 7f922ffff640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f921c06c9f0 0x7f921c06eeb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9224033000 tx=0x7f9224009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.613+0000 7faba9ad2640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fab8806c8f0 0x7fab8806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faba0000f60 tx=0x7faba0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.613+0000 7f9fa4ff9640 1 -- 172.21.15.40:0/686380992 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f9c01e400 con 0x7f9fa8151020 2024-04-25T11:38:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.614+0000 7f9fa4ff9640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f9f9406c940 0x7f9f9406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.614+0000 7f9fa4ff9640 1 -- 172.21.15.40:0/686380992 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9f9c090130 con 0x7f9fa8151020 2024-04-25T11:38:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.614+0000 7f9fa7fff640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f9f9406c940 0x7f9f9406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.614+0000 7f9fa7fff640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f9f9406c940 0x7f9f9406ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00072a0 tx=0x7f9fa00073b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.616+0000 7f6e377fe640 1 -- 172.21.15.40:0/2888741856 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e40057a00 con 0x7f6e480d9090 2024-04-25T11:38:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.616+0000 7f922cff9640 1 -- 172.21.15.40:0/622789275 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9230155540 con 0x7f92300cbaf0 2024-04-25T11:38:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.621+0000 7f6e377fe640 1 -- 172.21.15.40:0/2888741856 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e40022030 con 0x7f6e480d9090 2024-04-25T11:38:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.622+0000 7f9757fff640 1 -- 172.21.15.40:0/1286936063 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760041020 con 0x7f97700c83c0 2024-04-25T11:38:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.623+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97701553a0 con 0x7f97700c83c0 2024-04-25T11:38:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.623+0000 7fd6897fa640 1 -- 172.21.15.40:0/97903959 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684006b50 con 0x7fd6780051c0 2024-04-25T11:38:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.623+0000 7f48d0ff9640 1 -- 172.21.15.40:0/1132715742 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48c405a280 con 0x7f48d40707e0 2024-04-25T11:38:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.623+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c1ecf50 con 0x7fd6780051c0 2024-04-25T11:38:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.623+0000 7f03ed7fa640 1 -- 172.21.15.40:0/1475816348 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03f0153110 con 0x7f03f006f950 2024-04-25T11:38:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.623+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c14cea0 con 0x7fd6780051c0 2024-04-25T11:38:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.623+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9770154de0 con 0x7f97700c83c0 2024-04-25T11:38:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.624+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68c0c8e30 con 0x7fd6780051c0 2024-04-25T11:38:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.624+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977004fce0 con 0x7f97700c83c0 2024-04-25T11:38:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.632+0000 7f9757fff640 1 -- 172.21.15.40:0/1286936063 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9760032440 con 0x7f97700c83c0 2024-04-25T11:38:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.632+0000 7f9757fff640 1 -- 172.21.15.40:0/1286936063 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760037040 con 0x7f97700c83c0 2024-04-25T11:38:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.632+0000 7fab9a7fc640 1 -- 172.21.15.40:0/2185190765 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faba40ca470 con 0x7faba41eaa00 2024-04-25T11:38:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.632+0000 7fd6897fa640 1 -- 172.21.15.40:0/97903959 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68400cb60 con 0x7fd6780051c0 2024-04-25T11:38:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.632+0000 7fd6897fa640 1 -- 172.21.15.40:0/97903959 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68400cd10 con 0x7fd6780051c0 2024-04-25T11:38:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.633+0000 7f9fa4ff9640 1 -- 172.21.15.40:0/686380992 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9fa8143740 con 0x7f9fa8151020 2024-04-25T11:38:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.634+0000 7fd6897fa640 1 -- 172.21.15.40:0/97903959 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd68401a020 con 0x7fd6780051c0 2024-04-25T11:38:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.635+0000 7fd6897fa640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd66c06c8f0 0x7fd66c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.635+0000 7fd6897fa640 1 -- 172.21.15.40:0/97903959 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd68408c150 con 0x7fd6780051c0 2024-04-25T11:38:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.635+0000 7f9757fff640 1 -- 172.21.15.40:0/1286936063 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9760044020 con 0x7f97700c83c0 2024-04-25T11:38:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.636+0000 7f9757fff640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f975c06c8f0 0x7f975c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.636+0000 7fd68bfff640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd66c06c8f0 0x7fd66c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.636+0000 7f9757fff640 1 -- 172.21.15.40:0/1286936063 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f97600ac720 con 0x7f97700c83c0 2024-04-25T11:38:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.636+0000 7f976effd640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f975c06c8f0 0x7f975c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.636+0000 7fd68bfff640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd66c06c8f0 0x7fd66c06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd68c1e9d80 tx=0x7fd67c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.637+0000 7f976effd640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f975c06c8f0 0x7f975c06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97700cb6e0 tx=0x7f9764009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.643+0000 7fd6897fa640 1 -- 172.21.15.40:0/97903959 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd68c0c8e30 con 0x7fd6780051c0 2024-04-25T11:38:32.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:32.643+0000 7f9757fff640 1 -- 172.21.15.40:0/1286936063 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f976004b030 con 0x7f97700c83c0 2024-04-25T11:38:33.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.041+0000 7f92167fc640 1 -- 172.21.15.40:0/622789275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f923004fd50 con 0x7f92300cbaf0 2024-04-25T11:38:33.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.041+0000 7f922cff9640 1 -- 172.21.15.40:0/622789275 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f922005a2e0 con 0x7f92300cbaf0 2024-04-25T11:38:33.029 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-25T11:38:33.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.042+0000 7f48ba7fc640 1 -- 172.21.15.40:0/1132715742 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f48d40c7d70 con 0x7f48d40707e0 2024-04-25T11:38:33.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.043+0000 7f48d0ff9640 1 -- 172.21.15.40:0/1132715742 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48c4059b80 con 0x7f48d40707e0 2024-04-25T11:38:33.031 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-25T11:38:33.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.044+0000 7f92167fc640 1 -- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f921c06c9f0 msgr2=0x7f921c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.044+0000 7f92167fc640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f921c06c9f0 0x7f921c06eeb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9224033000 tx=0x7f9224009030 comp rx=0 tx=0).stop 2024-04-25T11:38:33.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.045+0000 7f92167fc640 1 -- 172.21.15.40:0/622789275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92300cbaf0 msgr2=0x7f92300c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.045+0000 7f92167fc640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92300cbaf0 0x7f92300c5800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f92300ca1a0 tx=0x7f922000d8a0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.045+0000 7f92167fc640 1 -- 172.21.15.40:0/622789275 shutdown_connections 2024-04-25T11:38:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.045+0000 7f92167fc640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f921c06c9f0 0x7f921c06eeb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.045+0000 7f92167fc640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92300cbaf0 0x7f92300c5800 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.045+0000 7f92167fc640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f923015fdc0 0x7f92300cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.045+0000 7f92167fc640 1 --2- 172.21.15.40:0/622789275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92180039b0 0x7f923014ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.046+0000 7f92167fc640 1 -- 172.21.15.40:0/622789275 >> 172.21.15.40:0/622789275 conn(0x7f923006f740 msgr2=0x7f9230157e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.046+0000 7f92167fc640 1 -- 172.21.15.40:0/622789275 shutdown_connections 2024-04-25T11:38:33.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.046+0000 7f92167fc640 1 -- 172.21.15.40:0/622789275 wait complete. 2024-04-25T11:38:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48c006c9c0 msgr2=0x7f48c006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48c006c9c0 0x7f48c006ee80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48d40cc1d0 tx=0x7f48c8009040 comp rx=0 tx=0).stop 2024-04-25T11:38:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d40707e0 msgr2=0x7f48d4155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d40707e0 0x7f48d4155380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e9b0 tx=0x7f48c400ee70 comp rx=0 tx=0).stop 2024-04-25T11:38:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 shutdown_connections 2024-04-25T11:38:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48c006c9c0 0x7f48c006ee80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d4151e40 0x7f48d41511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d4151a60 0x7f48d413d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 --2- 172.21.15.40:0/1132715742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d40707e0 0x7f48d4155380 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 >> 172.21.15.40:0/1132715742 conn(0x7f48d40cd170 msgr2=0x7f48d41559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 shutdown_connections 2024-04-25T11:38:33.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.047+0000 7f48da489640 1 -- 172.21.15.40:0/1132715742 wait complete. 2024-04-25T11:38:33.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.062+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6e30005580 con 0x7f6e480d9090 2024-04-25T11:38:33.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.062+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f03f006c740 con 0x7f03f006f950 2024-04-25T11:38:33.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.063+0000 7f03ed7fa640 1 -- 172.21.15.40:0/1475816348 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f03e404b030 con 0x7f03f006f950 2024-04-25T11:38:33.051 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-25T11:38:33.052 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-25T11:38:33.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.063+0000 7f6e377fe640 1 -- 172.21.15.40:0/2888741856 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e4005a060 con 0x7f6e480d9090 2024-04-25T11:38:33.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.063+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7faba404fce0 con 0x7faba41eaa00 2024-04-25T11:38:33.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.3 2024-04-25T11:38:33.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.064+0000 7fab9a7fc640 1 -- 172.21.15.40:0/2185190765 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fab9c07af50 con 0x7faba41eaa00 2024-04-25T11:38:33.054 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-25T11:38:33.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2024-04-25T11:38:33.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.065+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03d406c9c0 msgr2=0x7f03d406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.065+0000 7f03f6f20640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03d406c9c0 0x7f03d406ee80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03f015ca20 tx=0x7f03dc007950 comp rx=0 tx=0).stop 2024-04-25T11:38:33.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.065+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03f006f950 msgr2=0x7f03f014bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.065+0000 7f03f6f20640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03f006f950 0x7f03f014bc70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f03e4000c00 tx=0x7f03e40046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6e3c06c940 msgr2=0x7f6e3c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6e3c06c940 0x7f6e3c06ee00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098e0 tx=0x7f6e44008040 comp rx=0 tx=0).stop 2024-04-25T11:38:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 shutdown_connections 2024-04-25T11:38:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f03f6f20640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f03d406c9c0 0x7f03d406ee80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e480d9090 msgr2=0x7f6e4804e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f03f6f20640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03f01590c0 0x7f03f01464c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e480d9090 0x7f6e4804e610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000d580 tx=0x7f6e4000da40 comp rx=0 tx=0).stop 2024-04-25T11:38:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f03f6f20640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03f0072b70 0x7f03f01f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f03f6f20640 1 --2- 172.21.15.40:0/1475816348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03f006f950 0x7f03f014bc70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 >> 172.21.15.40:0/1475816348 conn(0x7f03f00cd200 msgr2=0x7f03f006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 shutdown_connections 2024-04-25T11:38:33.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6e3c06c940 0x7f6e3c06ee00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e480d9090 0x7f6e4804e610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e48050180 0x7f6e4804e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 --2- 172.21.15.40:0/2888741856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e48037160 0x7f6e48047ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 >> 172.21.15.40:0/2888741856 conn(0x7f6e48054610 msgr2=0x7f6e480d8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 shutdown_connections 2024-04-25T11:38:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f03f6f20640 1 -- 172.21.15.40:0/1475816348 wait complete. 2024-04-25T11:38:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 shutdown_connections 2024-04-25T11:38:33.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.066+0000 7f6e54822640 1 -- 172.21.15.40:0/2888741856 wait complete. 2024-04-25T11:38:33.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fab8806c8f0 msgr2=0x7fab8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fab8806c8f0 0x7fab8806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faba0000f60 tx=0x7faba0009040 comp rx=0 tx=0).stop 2024-04-25T11:38:33.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba41eaa00 msgr2=0x7faba40726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba41eaa00 0x7faba40726c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009d90 tx=0x7fab9c004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:33.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 shutdown_connections 2024-04-25T11:38:33.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fab8806c8f0 0x7fab8806edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba41eaa00 0x7faba40726c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faba4158f80 0x7faba41f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 --2- 172.21.15.40:0/2185190765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faba406e3a0 0x7faba4071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 >> 172.21.15.40:0/2185190765 conn(0x7faba40cc0d0 msgr2=0x7faba40af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 shutdown_connections 2024-04-25T11:38:33.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.068+0000 7fabaaad4640 1 -- 172.21.15.40:0/2185190765 wait complete. 2024-04-25T11:38:33.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.075+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9f88005850 con 0x7f9fa8151020 2024-04-25T11:38:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.075+0000 7f9fa4ff9640 1 -- 172.21.15.40:0/686380992 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9fa81ed060 con 0x7f9fa8151020 2024-04-25T11:38:33.063 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-25T11:38:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.075+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f97701fd430 con 0x7f97700c83c0 2024-04-25T11:38:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.076+0000 7f9757fff640 1 -- 172.21.15.40:0/1286936063 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f976007a600 con 0x7f97700c83c0 2024-04-25T11:38:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.076+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd68c04fd50 con 0x7fd6780051c0 2024-04-25T11:38:33.064 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-25T11:38:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.076+0000 7fd6897fa640 1 -- 172.21.15.40:0/97903959 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd68405a5f0 con 0x7fd6780051c0 2024-04-25T11:38:33.065 INFO:teuthology.orchestra.run.smithi040.stdout:55834574851 2024-04-25T11:38:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f9f9406c940 msgr2=0x7f9f9406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f9f9406c940 0x7f9f9406ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00072a0 tx=0x7f9fa00073b0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f975c06c8f0 msgr2=0x7f975c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f975c06c8f0 0x7f975c06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97700cb6e0 tx=0x7f9764009040 comp rx=0 tx=0).stop 2024-04-25T11:38:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa8151020 msgr2=0x7f9fa80ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa8151020 0x7f9fa80ca7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00ef00 tx=0x7f9f9c010e00 comp rx=0 tx=0).stop 2024-04-25T11:38:33.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97700c83c0 msgr2=0x7f97701fcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97700c83c0 0x7f97701fcef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f976002a7c0 tx=0x7f97600046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 shutdown_connections 2024-04-25T11:38:33.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f9f9406c940 0x7f9f9406ee00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fa8151020 0x7f9fa80ca7a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa80c8bd0 0x7f9fa8143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 shutdown_connections 2024-04-25T11:38:33.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 --2- 172.21.15.40:0/686380992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa8072440 0x7f9fa80cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f975c06c8f0 0x7f975c06edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9770152a40 0x7f97701635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 >> 172.21.15.40:0/686380992 conn(0x7f9fa806b970 msgr2=0x7f9fa806bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9770152660 0x7f97700caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 --2- 172.21.15.40:0/1286936063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97700c83c0 0x7f97701fcef0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 >> 172.21.15.40:0/1286936063 conn(0x7f977006d2e0 msgr2=0x7f9770157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 shutdown_connections 2024-04-25T11:38:33.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9fad638640 1 -- 172.21.15.40:0/686380992 wait complete. 2024-04-25T11:38:33.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 shutdown_connections 2024-04-25T11:38:33.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.079+0000 7f9775ac2640 1 -- 172.21.15.40:0/1286936063 wait complete. 2024-04-25T11:38:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd66c06c8f0 msgr2=0x7fd66c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd66c06c8f0 0x7fd66c06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd68c1e9d80 tx=0x7fd67c009190 comp rx=0 tx=0).stop 2024-04-25T11:38:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 msgr2=0x7fd68c1597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd68c1597d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd68400ebd0 tx=0x7fd68400c4b0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 shutdown_connections 2024-04-25T11:38:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd66c06c8f0 0x7fd66c06edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd68c14bfa0 0x7fd68c0cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd68c148e10 0x7fd68c14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 --2- 172.21.15.40:0/97903959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6780051c0 0x7fd68c1597d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 >> 172.21.15.40:0/97903959 conn(0x7fd68c06f740 msgr2=0x7fd68c149760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 shutdown_connections 2024-04-25T11:38:33.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:33.080+0000 7fd69317d640 1 -- 172.21.15.40:0/97903959 wait complete. 2024-04-25T11:38:33.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2024-04-25T11:38:33.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2024-04-25T11:38:33.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2024-04-25T11:38:33.081 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-04-25T11:38:33.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2024-04-25T11:38:33.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.6 2024-04-25T11:38:34.054 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:38:34.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:38:34.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T11:38:34.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T11:38:34.077 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:38:34.082 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T11:38:34.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:38:34.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T11:38:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.235+0000 7fbbcec59640 1 Processor -- start 2024-04-25T11:38:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.235+0000 7fbbcec59640 1 -- start start 2024-04-25T11:38:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.236+0000 7fbbcec59640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbbc8154ff0 0x7fbbc815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.237+0000 7fbbcc9f5640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbbc8154ff0 0x7fbbc815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.237+0000 7fbbc7fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbbc80b3fc0 0x7fbbc80cb480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:38274/0 (socket says 172.21.15.40:38274) 2024-04-25T11:38:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.237+0000 7fbbcc9f5640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbbc8154ff0 0x7fbbc815b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:52120/0 (socket says 172.21.15.40:52120) 2024-04-25T11:38:34.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.237+0000 7fbbcd1f6640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbbc815b9e0 0x7fbbc80d04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44368/0 (socket says 172.21.15.40:44368) 2024-04-25T11:38:34.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.237+0000 7fbbcec59640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc80cbfc0 con 0x7fbbc815b9e0 2024-04-25T11:38:34.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.237+0000 7fbbcec59640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc80cc280 con 0x7fbbc80b3fc0 2024-04-25T11:38:34.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.237+0000 7fbbcec59640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fbbc80cc560 con 0x7fbbc8154ff0 2024-04-25T11:38:34.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.237+0000 7fbbc7fff640 1 -- 172.21.15.40:0/1382376347 learned_addr learned my addr 172.21.15.40:0/1382376347 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:34.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.236+0000 7f7ec3067640 1 Processor -- start 2024-04-25T11:38:34.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.236+0000 7f7ec3067640 1 -- start start 2024-04-25T11:38:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.238+0000 7f7ec3067640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7ebc157b20 0x7f7ebc06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.238+0000 7f7ec3067640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ebc0c7b00 0x7f7ebc0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.239+0000 7fbbcc9f5640 1 -- 172.21.15.40:0/1382376347 >> v1:172.21.15.62:6789/0 conn(0x7fbbc80b3fc0 legacy=0x7fbbc80cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.239+0000 7fbbcc9f5640 1 -- 172.21.15.40:0/1382376347 >> v1:172.21.15.40:6789/0 conn(0x7fbbc815b9e0 legacy=0x7fbbc80d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.239+0000 7fbbcc9f5640 1 -- 172.21.15.40:0/1382376347 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc80cc840 con 0x7fbbc8154ff0 2024-04-25T11:38:34.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.238+0000 7f7ec3067640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ebc06bbc0 con 0x7f7ebc151160 2024-04-25T11:38:34.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.238+0000 7f7ec3067640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7ebc06b970 con 0x7f7ebc0c7b00 2024-04-25T11:38:34.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.238+0000 7f7ec3067640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f7ebc06bea0 con 0x7f7ebc157b20 2024-04-25T11:38:34.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.238+0000 7f7ec1604640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ebc0c7b00 0x7f7ebc0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.238+0000 7f7ec1604640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ebc0c7b00 0x7f7ebc0c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53382/0 (socket says 172.21.15.40:53382) 2024-04-25T11:38:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.238+0000 7f7ec1604640 1 -- 172.21.15.40:0/563392843 learned_addr learned my addr 172.21.15.40:0/563392843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.239+0000 7f7ec1604640 1 -- 172.21.15.40:0/563392843 >> v2:172.21.15.62:3300/0 conn(0x7f7ebc157b20 msgr2=0x7f7ebc06cc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:38:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.239+0000 7f7ec1604640 1 --2- 172.21.15.40:0/563392843 >> v2:172.21.15.62:3300/0 conn(0x7f7ebc157b20 0x7f7ebc06cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.239+0000 7f7ec1604640 1 -- 172.21.15.40:0/563392843 >> v1:172.21.15.62:6789/0 conn(0x7f7ebc151160 legacy=0x7f7ebc0c88f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:38:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.239+0000 7f7ec1604640 1 -- 172.21.15.40:0/563392843 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc06c180 con 0x7f7ebc0c7b00 2024-04-25T11:38:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.240+0000 7f7ec1604640 1 --2- 172.21.15.40:0/563392843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc0c7b00 0x7f7ebc0c7f10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc071550 tx=0x7f7eb402cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298546c933bffefd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.241+0000 7f7ebb7fe640 1 -- 172.21.15.40:0/563392843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb403b070 con 0x7f7ebc0c7b00 2024-04-25T11:38:34.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.241+0000 7f7ebb7fe640 1 -- 172.21.15.40:0/563392843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb40090d0 con 0x7f7ebc0c7b00 2024-04-25T11:38:34.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.242+0000 7f7ec3067640 1 -- 172.21.15.40:0/563392843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc0c7b00 msgr2=0x7f7ebc0c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.242+0000 7f7ec3067640 1 --2- 172.21.15.40:0/563392843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc0c7b00 0x7f7ebc0c7f10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc071550 tx=0x7f7eb402cc10 comp rx=0 tx=0).stop 2024-04-25T11:38:34.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7f7ec3067640 1 -- 172.21.15.40:0/563392843 shutdown_connections 2024-04-25T11:38:34.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7fbbcc9f5640 1 --2- 172.21.15.40:0/1382376347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8154ff0 0x7fbbc815b4a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0097a0 tx=0x7fbbbc02a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2dc8dfdf090a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7f7ec3067640 1 --2- 172.21.15.40:0/563392843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc0c7b00 0x7f7ebc0c7f10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7f7ec3067640 1 --2- 172.21.15.40:0/563392843 >> v2:172.21.15.62:3300/0 conn(0x7f7ebc157b20 0x7f7ebc06cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7f7ec3067640 1 -- 172.21.15.40:0/563392843 >> 172.21.15.40:0/563392843 conn(0x7f7ebc0c94b0 msgr2=0x7f7ebc0c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7f7ec3067640 1 -- 172.21.15.40:0/563392843 shutdown_connections 2024-04-25T11:38:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7f7ec3067640 1 -- 172.21.15.40:0/563392843 wait complete. 2024-04-25T11:38:34.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7fbbc77fe640 1 -- 172.21.15.40:0/1382376347 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc037070 con 0x7fbbc8154ff0 2024-04-25T11:38:34.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7fbbc77fe640 1 -- 172.21.15.40:0/1382376347 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbbc02acd0 con 0x7fbbc8154ff0 2024-04-25T11:38:34.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.243+0000 7fbbc77fe640 1 -- 172.21.15.40:0/1382376347 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc032660 con 0x7fbbc8154ff0 2024-04-25T11:38:34.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.245+0000 7fbbcec59640 1 -- 172.21.15.40:0/1382376347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8154ff0 msgr2=0x7fbbc815b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.245+0000 7fbbcec59640 1 --2- 172.21.15.40:0/1382376347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8154ff0 0x7fbbc815b4a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0097a0 tx=0x7fbbbc02a040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.245+0000 7f7ec3067640 1 Processor -- start 2024-04-25T11:38:34.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.245+0000 7f7ec3067640 1 -- start start 2024-04-25T11:38:34.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.245+0000 7fbbcec59640 1 -- 172.21.15.40:0/1382376347 shutdown_connections 2024-04-25T11:38:34.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.245+0000 7fbbcec59640 1 --2- 172.21.15.40:0/1382376347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8154ff0 0x7fbbc815b4a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.245+0000 7fbbcec59640 1 -- 172.21.15.40:0/1382376347 >> 172.21.15.40:0/1382376347 conn(0x7fbbc806c680 msgr2=0x7fbbc806ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.245+0000 7fbbcec59640 1 -- 172.21.15.40:0/1382376347 shutdown_connections 2024-04-25T11:38:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.246+0000 7f7ec3067640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ebc0c7b00 0x7f7ebc1f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.246+0000 7f7ec3067640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc151160 0x7f7ebc0725e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.247+0000 7f7ec0e03640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ebc0c7b00 0x7f7ebc1f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.247+0000 7f7ec0e03640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ebc0c7b00 0x7f7ebc1f86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:38166/0 (socket says 172.21.15.40:38166) 2024-04-25T11:38:34.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.246+0000 7fbbcec59640 1 -- 172.21.15.40:0/1382376347 wait complete. 2024-04-25T11:38:34.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.247+0000 7fbbcec59640 1 Processor -- start 2024-04-25T11:38:34.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.247+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc151160 0x7f7ebc0725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.248+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc151160 0x7f7ebc0725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53398/0 (socket says 172.21.15.40:53398) 2024-04-25T11:38:34.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.248+0000 7f7ec0e03640 1 -- 172.21.15.40:0/2889937963 learned_addr learned my addr 172.21.15.40:0/2889937963 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.248+0000 7f7ec3067640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc157b20 0x7f7ebc072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.248+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc151540 con 0x7f7ebc151160 2024-04-25T11:38:34.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.248+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc06c740 con 0x7f7ebc157b20 2024-04-25T11:38:34.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.248+0000 7f7ec1604640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc157b20 0x7f7ebc072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.248+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7ebc157f80 con 0x7f7ebc0c7b00 2024-04-25T11:38:34.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.247+0000 7fbbcec59640 1 -- start start 2024-04-25T11:38:34.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.249+0000 7f7ebbfff640 1 -- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ebc0c7b00 msgr2=0x7f7ebc1f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.249+0000 7f7ebbfff640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ebc0c7b00 0x7f7ebc1f86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.249+0000 7f7ebbfff640 1 -- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc157b20 msgr2=0x7f7ebc072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.249+0000 7f7ebbfff640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc157b20 0x7f7ebc072b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.249+0000 7f7ebbfff640 1 -- 172.21.15.40:0/2889937963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc1f9100 con 0x7f7ebc151160 2024-04-25T11:38:34.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbcec59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80b3fc0 0x7fbbc806f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbcec59640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc8154ff0 0x7fbbc80c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7f7ebbfff640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc151160 0x7f7ebc0725e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00b7b0 tx=0x7f7eac00bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7f7eb97fa640 1 -- 172.21.15.40:0/2889937963 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac019070 con 0x7f7ebc151160 2024-04-25T11:38:34.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7f7eb97fa640 1 -- 172.21.15.40:0/2889937963 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eac00c740 con 0x7f7ebc151160 2024-04-25T11:38:34.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbcec59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbc815b9e0 0x7fbbc815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbcec59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81553d0 con 0x7fbbc815b9e0 2024-04-25T11:38:34.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbc7fff640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc8154ff0 0x7fbbc80c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbcec59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc815af80 con 0x7fbbc80b3fc0 2024-04-25T11:38:34.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbcec59640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc81fde80 con 0x7fbbc8154ff0 2024-04-25T11:38:34.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbc7fff640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc8154ff0 0x7fbbc80c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:38182/0 (socket says 172.21.15.40:38182) 2024-04-25T11:38:34.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbc7fff640 1 -- 172.21.15.40:0/1199407017 learned_addr learned my addr 172.21.15.40:0/1199407017 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.250+0000 7fbbcd1f6640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbc815b9e0 0x7fbbc815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.251+0000 7fbbc7fff640 1 -- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80b3fc0 msgr2=0x7fbbc806f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.251+0000 7fbbc7fff640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80b3fc0 0x7fbbc806f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.251+0000 7fbbc7fff640 1 -- 172.21.15.40:0/1199407017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbc815b9e0 msgr2=0x7fbbc815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.251+0000 7fbbc7fff640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbc815b9e0 0x7fbbc815e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.251+0000 7fbbc7fff640 1 -- 172.21.15.40:0/1199407017 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc815f130 con 0x7fbbc8154ff0 2024-04-25T11:38:34.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.251+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc1485f0 con 0x7f7ebc151160 2024-04-25T11:38:34.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.251+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc148bb0 con 0x7f7ebc151160 2024-04-25T11:38:34.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.251+0000 7fbbc7fff640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc8154ff0 0x7fbbc80c4310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400c540 tx=0x7fbbb400ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.251+0000 7f7eb97fa640 1 -- 172.21.15.40:0/2889937963 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00cb30 con 0x7f7ebc151160 2024-04-25T11:38:34.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.252+0000 7fbbc57fa640 1 -- 172.21.15.40:0/1199407017 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4018070 con 0x7fbbc8154ff0 2024-04-25T11:38:34.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.252+0000 7fbbc57fa640 1 -- 172.21.15.40:0/1199407017 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb40074e0 con 0x7fbbc8154ff0 2024-04-25T11:38:34.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.252+0000 7fbbcec59640 1 -- 172.21.15.40:0/1199407017 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc815d380 con 0x7fbbc8154ff0 2024-04-25T11:38:34.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.252+0000 7fbbc57fa640 1 -- 172.21.15.40:0/1199407017 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40049e0 con 0x7fbbc8154ff0 2024-04-25T11:38:34.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.252+0000 7fbbcec59640 1 -- 172.21.15.40:0/1199407017 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc815d940 con 0x7fbbc8154ff0 2024-04-25T11:38:34.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.254+0000 7f7eb97fa640 1 -- 172.21.15.40:0/2889937963 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7eac01c070 con 0x7f7ebc151160 2024-04-25T11:38:34.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.255+0000 7f7eb97fa640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7ea806c8f0 0x7f7ea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.255+0000 7f7eb97fa640 1 -- 172.21.15.40:0/2889937963 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7eac08b0c0 con 0x7f7ebc151160 2024-04-25T11:38:34.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.255+0000 7f7ec0e03640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7ea806c8f0 0x7f7ea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.256+0000 7fbbc57fa640 1 -- 172.21.15.40:0/1199407017 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbb401b070 con 0x7fbbc8154ff0 2024-04-25T11:38:34.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.256+0000 7fbbc57fa640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbbac06c9c0 0x7fbbac06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.256+0000 7fbbcec59640 1 -- 172.21.15.40:0/1199407017 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba8005290 con 0x7fbbc8154ff0 2024-04-25T11:38:34.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.256+0000 7fbbc57fa640 1 -- 172.21.15.40:0/1199407017 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbbb408afe0 con 0x7fbbc8154ff0 2024-04-25T11:38:34.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.257+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc148df0 con 0x7f7ebc151160 2024-04-25T11:38:34.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.257+0000 7fbbcc9f5640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbbac06c9c0 0x7fbbac06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.259+0000 7f7ec0e03640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7ea806c8f0 0x7f7ea806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006fd0 tx=0x7f7eb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.260+0000 7fbbcc9f5640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbbac06c9c0 0x7fbbac06ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc034000 tx=0x7fbbbc02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.265+0000 7fbbc57fa640 1 -- 172.21.15.40:0/1199407017 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbbb4058d10 con 0x7fbbc8154ff0 2024-04-25T11:38:34.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.267+0000 7f7eb97fa640 1 -- 172.21.15.40:0/2889937963 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ebc148df0 con 0x7f7ebc151160 2024-04-25T11:38:34.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.286+0000 7f8ac1eef640 1 Processor -- start 2024-04-25T11:38:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.286+0000 7f8ac1eef640 1 -- start start 2024-04-25T11:38:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.288+0000 7f8ac1eef640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8abc072a10 0x7f8abc1626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.288+0000 7f8ac1eef640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc072df0 0x7f8abc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.288+0000 7f8ac1eef640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8abc06c890 con 0x7f8abc0c8940 2024-04-25T11:38:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.288+0000 7f8ac1eef640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8abc0c84b0 con 0x7f8abc072a10 2024-04-25T11:38:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.288+0000 7f8ac1eef640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc162ce0 con 0x7f8abc072df0 2024-04-25T11:38:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7f8abbfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8abc0c8940 0x7f8abc158e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44376/0 (socket says 172.21.15.40:44376) 2024-04-25T11:38:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7f8abbfff640 1 -- 172.21.15.40:0/3126559057 learned_addr learned my addr 172.21.15.40:0/3126559057 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7fe409a69640 1 Processor -- start 2024-04-25T11:38:34.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7f0fa34cf640 1 Processor -- start 2024-04-25T11:38:34.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7fe409a69640 1 -- start start 2024-04-25T11:38:34.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7f8abaffd640 1 --2- 172.21.15.40:0/3126559057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc072df0 0x7f8abc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7f0fa34cf640 1 -- start start 2024-04-25T11:38:34.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7f8abaffd640 1 -- 172.21.15.40:0/3126559057 >> v2:172.21.15.40:3300/0 conn(0x7f8abc072a10 msgr2=0x7f8abc1626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7f8abaffd640 1 --2- 172.21.15.40:0/3126559057 >> v2:172.21.15.40:3300/0 conn(0x7f8abc072a10 0x7f8abc1626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7f8abaffd640 1 -- 172.21.15.40:0/3126559057 >> v1:172.21.15.40:6789/0 conn(0x7f8abc0c8940 legacy=0x7f8abc158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.289+0000 7f8abaffd640 1 -- 172.21.15.40:0/3126559057 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc162fc0 con 0x7f8abc072df0 2024-04-25T11:38:34.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.290+0000 7f0fa34cf640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c06c490 0x7f0f9c072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.290+0000 7f0fa34cf640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0f9c072900 0x7f0f9c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.290+0000 7f0fa34cf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f9c157da0 con 0x7f0f9c151040 2024-04-25T11:38:34.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.290+0000 7f8abaffd640 1 --2- 172.21.15.40:0/3126559057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc072df0 0x7f8abc06c350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009d90 tx=0x7f8ab402a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c25ef8b8cc570a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.290+0000 7f0fa34cf640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f0f9c0c86c0 con 0x7f0f9c072900 2024-04-25T11:38:34.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.290+0000 7f0fa34cf640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9c1514d0 con 0x7f0f9c06c490 2024-04-25T11:38:34.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f0fa0a6a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0f9c072900 0x7f0f9c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7fb7f6cfd640 1 Processor -- start 2024-04-25T11:38:34.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f0fa126b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c06c490 0x7f0f9c072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7fb7f6cfd640 1 -- start start 2024-04-25T11:38:34.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7fe408a67640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe404151180 0x7fe40406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44400/0 (socket says 172.21.15.40:44400) 2024-04-25T11:38:34.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7fe408a67640 1 -- 172.21.15.40:0/1336158905 learned_addr learned my addr 172.21.15.40:0/1336158905 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:34.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f0fa0a6a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0f9c072900 0x7f0f9c156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:52152/0 (socket says 172.21.15.40:52152) 2024-04-25T11:38:34.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f0fa0a6a640 1 -- 172.21.15.40:0/2205761341 learned_addr learned my addr 172.21.15.40:0/2205761341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7fe409a69640 1 --2- 172.21.15.40:0/1336158905 >> v2:172.21.15.40:3300/0 conn(0x7fe4040c7340 0x7fe4040c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f8aba7fc640 1 -- 172.21.15.40:0/3126559057 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4007740 con 0x7f8abc072df0 2024-04-25T11:38:34.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f8aba7fc640 1 -- 172.21.15.40:0/3126559057 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab40078e0 con 0x7f8abc072df0 2024-04-25T11:38:34.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f8aba7fc640 1 -- 172.21.15.40:0/3126559057 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4007cf0 con 0x7f8abc072df0 2024-04-25T11:38:34.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7fe409a69640 1 -- 172.21.15.40:0/1336158905 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe40406c420 con 0x7fe404151180 2024-04-25T11:38:34.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7fe409a69640 1 -- 172.21.15.40:0/1336158905 --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4040c8b50 con 0x7fe40415a570 2024-04-25T11:38:34.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7fe409a69640 1 -- 172.21.15.40:0/1336158905 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe40406c750 con 0x7fe4040c7340 2024-04-25T11:38:34.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7fe403fff640 1 --2- 172.21.15.40:0/1336158905 >> v2:172.21.15.40:3300/0 conn(0x7fe4040c7340 0x7fe4040c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7f8ac1eef640 1 -- 172.21.15.40:0/3126559057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc072df0 msgr2=0x7f8abc06c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7f8ac1eef640 1 --2- 172.21.15.40:0/3126559057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc072df0 0x7f8abc06c350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009d90 tx=0x7f8ab402a630 comp rx=0 tx=0).stop 2024-04-25T11:38:34.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7fb7f6cfd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7e8046310 0x7fb7e80df010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7fb7f6cfd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e8046760 0x7fb7e80df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7fb7f4a99640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb7e80d5cb0 0x7fb7e804c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44404/0 (socket says 172.21.15.40:44404) 2024-04-25T11:38:34.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7f8ac1eef640 1 -- 172.21.15.40:0/3126559057 shutdown_connections 2024-04-25T11:38:34.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7f8ac1eef640 1 --2- 172.21.15.40:0/3126559057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc072df0 0x7f8abc06c350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7f8ac1eef640 1 --2- 172.21.15.40:0/3126559057 >> v2:172.21.15.40:3300/0 conn(0x7f8abc072a10 0x7f8abc1626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7f8ac1eef640 1 -- 172.21.15.40:0/3126559057 >> 172.21.15.40:0/3126559057 conn(0x7f8abc0cd170 msgr2=0x7f8abc0cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f0fa126b640 1 -- 172.21.15.40:0/2205761341 >> v2:172.21.15.62:3300/0 conn(0x7f0f9c072900 msgr2=0x7f0f9c156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f0fa126b640 1 --2- 172.21.15.40:0/2205761341 >> v2:172.21.15.62:3300/0 conn(0x7f0f9c072900 0x7f0f9c156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.291+0000 7f0fa126b640 1 -- 172.21.15.40:0/2205761341 >> v1:172.21.15.40:6789/0 conn(0x7f0f9c151040 legacy=0x7f0f9c0c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7f0fa126b640 1 -- 172.21.15.40:0/2205761341 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c06eaf0 con 0x7f0f9c06c490 2024-04-25T11:38:34.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7f8ac1eef640 1 -- 172.21.15.40:0/3126559057 shutdown_connections 2024-04-25T11:38:34.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7fb7f6cfd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e804d3c0 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7fb7f6cfd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e804c9d0 con 0x7fb7e8046310 2024-04-25T11:38:34.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7fb7f6cfd640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e8008610 con 0x7fb7e8046760 2024-04-25T11:38:34.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7fb7f4a99640 1 -- 172.21.15.40:0/2109300056 learned_addr learned my addr 172.21.15.40:0/2109300056 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:34.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7fb7f529a640 1 --2- 172.21.15.40:0/2109300056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e8046760 0x7fb7e80df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7fe403fff640 1 -- 172.21.15.40:0/1336158905 >> v1:172.21.15.62:6789/0 conn(0x7fe40415a570 legacy=0x7fe40415a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.292+0000 7f8ac1eef640 1 -- 172.21.15.40:0/3126559057 wait complete. 2024-04-25T11:38:34.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7f0fa126b640 1 --2- 172.21.15.40:0/2205761341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c06c490 0x7f0f9c072ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0f980097a0 tx=0x7f0f9802a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb3d1d14858f4663 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7fb7effff640 1 --2- 172.21.15.40:0/2109300056 >> v2:172.21.15.40:3300/0 conn(0x7fb7e8046310 0x7fb7e80df010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7fb7f529a640 1 -- 172.21.15.40:0/2109300056 >> v2:172.21.15.40:3300/0 conn(0x7fb7e8046310 msgr2=0x7fb7e80df010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7fb7f529a640 1 --2- 172.21.15.40:0/2109300056 >> v2:172.21.15.40:3300/0 conn(0x7fb7e8046310 0x7fb7e80df010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7fb7f529a640 1 -- 172.21.15.40:0/2109300056 >> v1:172.21.15.40:6789/0 conn(0x7fb7e80d5cb0 legacy=0x7fb7e804c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7fb7f529a640 1 -- 172.21.15.40:0/2109300056 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e804d680 con 0x7fb7e8046760 2024-04-25T11:38:34.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7fb7effff640 1 --2- 172.21.15.40:0/2109300056 >> v2:172.21.15.40:3300/0 conn(0x7fb7e8046310 0x7fb7e80df010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7fe403fff640 1 -- 172.21.15.40:0/1336158905 >> v1:172.21.15.40:6789/0 conn(0x7fe404151180 legacy=0x7fe40406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7fe403fff640 1 -- 172.21.15.40:0/1336158905 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4040c8e30 con 0x7fe4040c7340 2024-04-25T11:38:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7f0f93fff640 1 -- 172.21.15.40:0/2205761341 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98037070 con 0x7f0f9c06c490 2024-04-25T11:38:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7f0f93fff640 1 -- 172.21.15.40:0/2205761341 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f98032440 con 0x7f0f9c06c490 2024-04-25T11:38:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.293+0000 7f0f93fff640 1 -- 172.21.15.40:0/2205761341 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98032850 con 0x7f0f9c06c490 2024-04-25T11:38:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.294+0000 7f89bb920640 1 Processor -- start 2024-04-25T11:38:34.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.294+0000 7fe403fff640 1 --2- 172.21.15.40:0/1336158905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4040c7340 0x7fe4040c7c50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000ddf0 tx=0x7fe3f0032b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28660a3564b900b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.294+0000 7fe4037fe640 1 -- 172.21.15.40:0/1336158905 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0034860 con 0x7fe4040c7340 2024-04-25T11:38:34.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.294+0000 7fe4037fe640 1 -- 172.21.15.40:0/1336158905 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f0034a00 con 0x7fe4040c7340 2024-04-25T11:38:34.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.294+0000 7fe4037fe640 1 -- 172.21.15.40:0/1336158905 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0034dd0 con 0x7fe4040c7340 2024-04-25T11:38:34.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7f0fa34cf640 1 -- 172.21.15.40:0/2205761341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c06c490 msgr2=0x7f0f9c072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.294+0000 7f8ac1eef640 1 Processor -- start 2024-04-25T11:38:34.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.294+0000 7f8ac1eef640 1 -- start start 2024-04-25T11:38:34.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7f0fa34cf640 1 --2- 172.21.15.40:0/2205761341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c06c490 0x7f0f9c072ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0f980097a0 tx=0x7f0f9802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.294+0000 7f89bb920640 1 -- start start 2024-04-25T11:38:34.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7fb7f529a640 1 --2- 172.21.15.40:0/2109300056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e8046760 0x7fb7e80df550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb7f002db50 tx=0x7fb7f00a62e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd140b3669a4954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7f0fa34cf640 1 -- 172.21.15.40:0/2205761341 shutdown_connections 2024-04-25T11:38:34.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7f0fa34cf640 1 --2- 172.21.15.40:0/2205761341 >> v2:172.21.15.62:3300/0 conn(0x7f0f9c072900 0x7f0f9c156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7f0fa34cf640 1 --2- 172.21.15.40:0/2205761341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c06c490 0x7f0f9c072ef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7f0fa34cf640 1 -- 172.21.15.40:0/2205761341 >> 172.21.15.40:0/2205761341 conn(0x7f0f9c06b970 msgr2=0x7f0f9c06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7fe409a69640 1 -- 172.21.15.40:0/1336158905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4040c7340 msgr2=0x7fe4040c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7fe409a69640 1 --2- 172.21.15.40:0/1336158905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4040c7340 0x7fe4040c7c50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000ddf0 tx=0x7fe3f0032b40 comp rx=0 tx=0).stop 2024-04-25T11:38:34.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7f0fa34cf640 1 -- 172.21.15.40:0/2205761341 shutdown_connections 2024-04-25T11:38:34.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7f0fa34cf640 1 -- 172.21.15.40:0/2205761341 wait complete. 2024-04-25T11:38:34.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.295+0000 7f8ac1eef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8abc072a10 0x7f8abc150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7fe409a69640 1 -- 172.21.15.40:0/1336158905 shutdown_connections 2024-04-25T11:38:34.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f8ac1eef640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc072df0 0x7f8abc0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7fe409a69640 1 --2- 172.21.15.40:0/1336158905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4040c7340 0x7fe4040c7c50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7fe409a69640 1 -- 172.21.15.40:0/1336158905 >> 172.21.15.40:0/1336158905 conn(0x7fe4040cc940 msgr2=0x7fe4040ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8abc072a10 0x7f8abc150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7fe409a69640 1 -- 172.21.15.40:0/1336158905 shutdown_connections 2024-04-25T11:38:34.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f8abb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8abc072a10 0x7f8abc150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53434/0 (socket says 172.21.15.40:53434) 2024-04-25T11:38:34.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7fe409a69640 1 -- 172.21.15.40:0/1336158905 wait complete. 2024-04-25T11:38:34.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f8ac1eef640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc0c8940 0x7f8abc161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f89bb920640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b4159370 0x7f89b41626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc072df0 0x7f8abc0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc072df0 0x7f8abc0cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:52156/0 (socket says 172.21.15.40:52156) 2024-04-25T11:38:34.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f8ac1eef640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0c8140 con 0x7f8abc072a10 2024-04-25T11:38:34.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f8ac1eef640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc163580 con 0x7f8abc072df0 2024-04-25T11:38:34.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f8ac1eef640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc0733f0 con 0x7f8abc0c8940 2024-04-25T11:38:34.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f89bb920640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89b406fe90 0x7f89b4071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7fb7ef7fe640 1 -- 172.21.15.40:0/2109300056 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00b3070 con 0x7fb7e8046760 2024-04-25T11:38:34.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7fb7ef7fe640 1 -- 172.21.15.40:0/2109300056 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f00ae640 con 0x7fb7e8046760 2024-04-25T11:38:34.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f89bb920640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b4072f20 con 0x7f89b4158f90 2024-04-25T11:38:34.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7fb7ef7fe640 1 -- 172.21.15.40:0/2109300056 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00aea70 con 0x7fb7e8046760 2024-04-25T11:38:34.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f89bb920640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f89b41600a0 con 0x7f89b406fe90 2024-04-25T11:38:34.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f89bb920640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f89b415f2e0 con 0x7f89b4159370 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f89b9ebd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89b406fe90 0x7f89b4071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f89b9ebd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89b406fe90 0x7f89b4071fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53446/0 (socket says 172.21.15.40:53446) 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f89b9ebd640 1 -- 172.21.15.40:0/2355841852 learned_addr learned my addr 172.21.15.40:0/2355841852 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.296+0000 7f89b8ebb640 1 --2- 172.21.15.40:0/2355841852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b4159370 0x7f89b41626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7f89b9ebd640 1 -- 172.21.15.40:0/2355841852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b4159370 msgr2=0x7f89b41626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7f89b9ebd640 1 --2- 172.21.15.40:0/2355841852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b4159370 0x7f89b41626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7f89b9ebd640 1 -- 172.21.15.40:0/2355841852 >> v1:172.21.15.40:6789/0 conn(0x7f89b4158f90 legacy=0x7f89b4071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7f89b9ebd640 1 -- 172.21.15.40:0/2355841852 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b415f5c0 con 0x7f89b406fe90 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc0c8940 0x7f8abc161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7f8abb7fe640 1 -- 172.21.15.40:0/771952280 learned_addr learned my addr 172.21.15.40:0/771952280 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/2109300056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e8046760 msgr2=0x7fb7e80df550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fb7f6cfd640 1 --2- 172.21.15.40:0/2109300056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e8046760 0x7fb7e80df550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb7f002db50 tx=0x7fb7f00a62e0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7f0fa34cf640 1 Processor -- start 2024-04-25T11:38:34.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.297+0000 7f0fa34cf640 1 -- start start 2024-04-25T11:38:34.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/2109300056 shutdown_connections 2024-04-25T11:38:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fb7f6cfd640 1 --2- 172.21.15.40:0/2109300056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e8046760 0x7fb7e80df550 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fb7f6cfd640 1 --2- 172.21.15.40:0/2109300056 >> v2:172.21.15.40:3300/0 conn(0x7fb7e8046310 0x7fb7e80df010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/2109300056 >> 172.21.15.40:0/2109300056 conn(0x7fb7e8051280 msgr2=0x7fb7e8051690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/2109300056 shutdown_connections 2024-04-25T11:38:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fe409a69640 1 Processor -- start 2024-04-25T11:38:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/2109300056 wait complete. 2024-04-25T11:38:34.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7fe409a69640 1 -- start start 2024-04-25T11:38:34.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7f8abaffd640 1 -- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc0c8940 msgr2=0x7f8abc161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7f8abaffd640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc0c8940 0x7f8abc161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7f8abaffd640 1 -- 172.21.15.40:0/771952280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8abc072a10 msgr2=0x7f8abc150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7f8abaffd640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8abc072a10 0x7f8abc150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7f8abaffd640 1 -- 172.21.15.40:0/771952280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc0cc260 con 0x7f8abc072df0 2024-04-25T11:38:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.298+0000 7f0fa34cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9c06c490 0x7f0f9c13b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa34cf640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c072900 0x7f0f9c165b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa34cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f9c151040 0x7f0f9c132c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa34cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c157ac0 con 0x7f0f9c06c490 2024-04-25T11:38:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa34cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1577e0 con 0x7f0f9c151040 2024-04-25T11:38:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa34cf640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9c0c77b0 con 0x7f0f9c072900 2024-04-25T11:38:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f8abaffd640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc072df0 0x7f8abc0cbb80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4031000 tx=0x7f8ab4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa126b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9c06c490 0x7f0f9c13b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa126b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9c06c490 0x7f0f9c13b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53452/0 (socket says 172.21.15.40:53452) 2024-04-25T11:38:34.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa126b640 1 -- 172.21.15.40:0/896732748 learned_addr learned my addr 172.21.15.40:0/896732748 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f89b9ebd640 1 --2- 172.21.15.40:0/2355841852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b406fe90 0x7f89b4071fa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009870 tx=0x7f89a402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f00b821546ebfa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa0a6a640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c072900 0x7f0f9c165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa1a6c640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f9c151040 0x7f0f9c132c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa126b640 1 -- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c072900 msgr2=0x7f0f9c165b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa126b640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c072900 0x7f0f9c165b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa126b640 1 -- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f9c151040 msgr2=0x7f0f9c132c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa126b640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f9c151040 0x7f0f9c132c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f0fa126b640 1 -- 172.21.15.40:0/896732748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c1660c0 con 0x7f0f9c06c490 2024-04-25T11:38:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7fe409a69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe404151180 0x7fe404146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.299+0000 7f8aa3fff640 1 -- 172.21.15.40:0/771952280 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4040070 con 0x7f8abc072df0 2024-04-25T11:38:34.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f8aa3fff640 1 -- 172.21.15.40:0/771952280 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab4031cd0 con 0x7f8abc072df0 2024-04-25T11:38:34.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f8aa3fff640 1 -- 172.21.15.40:0/771952280 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4045680 con 0x7f8abc072df0 2024-04-25T11:38:34.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe409a69640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40415a570 0x7fe404144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc1f5d70 con 0x7f8abc072df0 2024-04-25T11:38:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe408a67640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe404151180 0x7fe404146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f0fa126b640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9c06c490 0x7f0f9c13b280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f9802a7d0 tx=0x7f0f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc0cb380 con 0x7f8abc072df0 2024-04-25T11:38:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f89abfff640 1 -- 172.21.15.40:0/2355841852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4037070 con 0x7f89b406fe90 2024-04-25T11:38:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f89abfff640 1 -- 172.21.15.40:0/2355841852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a4032440 con 0x7f89b406fe90 2024-04-25T11:38:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f89abfff640 1 -- 172.21.15.40:0/2355841852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4032810 con 0x7f89b406fe90 2024-04-25T11:38:34.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe403fff640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40415a570 0x7fe404144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe403fff640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40415a570 0x7fe404144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:38240/0 (socket says 172.21.15.40:38240) 2024-04-25T11:38:34.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe408a67640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe404151180 0x7fe404146740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:52168/0 (socket says 172.21.15.40:52168) 2024-04-25T11:38:34.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f0fa1a6c640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f9c151040 0x7f0f9c132c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:34.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe409a69640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4041efcd0 0x7fe4041edc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe403fff640 1 -- 172.21.15.40:0/2872932162 learned_addr learned my addr 172.21.15.40:0/2872932162 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f0f91ffb640 1 -- 172.21.15.40:0/896732748 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98037050 con 0x7f0f9c06c490 2024-04-25T11:38:34.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe409268640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4041efcd0 0x7fe4041edc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7f0f91ffb640 1 -- 172.21.15.40:0/896732748 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f98038040 con 0x7f0f9c06c490 2024-04-25T11:38:34.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe404151560 con 0x7fe4041efcd0 2024-04-25T11:38:34.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe4040c3b30 con 0x7fe404151180 2024-04-25T11:38:34.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe4041516e0 con 0x7fe40415a570 2024-04-25T11:38:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c137e40 con 0x7f0f9c06c490 2024-04-25T11:38:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7fe403fff640 1 -- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe404151180 msgr2=0x7fe404146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7fe403fff640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe404151180 0x7fe404146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7fe403fff640 1 -- 172.21.15.40:0/2872932162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4041efcd0 msgr2=0x7fe4041edc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7fe403fff640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4041efcd0 0x7fe4041edc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7fe403fff640 1 -- 172.21.15.40:0/2872932162 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40414b6a0 con 0x7fe40415a570 2024-04-25T11:38:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7fe403fff640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40415a570 0x7fe404144680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000df10 tx=0x7fe3f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7f0f91ffb640 1 -- 172.21.15.40:0/896732748 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9803cb30 con 0x7f0f9c06c490 2024-04-25T11:38:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c13aa80 con 0x7f0f9c06c490 2024-04-25T11:38:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.301+0000 7fe4017fa640 1 -- 172.21.15.40:0/2872932162 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f003e400 con 0x7fe40415a570 2024-04-25T11:38:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7fe4017fa640 1 -- 172.21.15.40:0/2872932162 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f004d070 con 0x7fe40415a570 2024-04-25T11:38:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7fe4017fa640 1 -- 172.21.15.40:0/2872932162 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f003e7a0 con 0x7fe40415a570 2024-04-25T11:38:34.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc069a10 con 0x7f8abc072df0 2024-04-25T11:38:34.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4041f0670 con 0x7fe40415a570 2024-04-25T11:38:34.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe404143ef0 con 0x7fe40415a570 2024-04-25T11:38:34.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fb7f6cfd640 1 Processor -- start 2024-04-25T11:38:34.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.300+0000 7fb7f6cfd640 1 -- start start 2024-04-25T11:38:34.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7fb7f6cfd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8046310 0x7fb7e80d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7fb7f6cfd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e8046760 0x7fb7e8050840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7fb7f6cfd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e80d5cb0 0x7fb7e80e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7f8aa3fff640 1 -- 172.21.15.40:0/771952280 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ab4039040 con 0x7f8abc072df0 2024-04-25T11:38:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.302+0000 7f8aa3fff640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8aa806c9f0 0x7f8aa806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7f8aa3fff640 1 -- 172.21.15.40:0/771952280 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ab40ad2d0 con 0x7f8abc072df0 2024-04-25T11:38:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7fb7f6cfd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e8008b40 con 0x7fb7e8046760 2024-04-25T11:38:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7fb7f6cfd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e8008860 con 0x7fb7e8046310 2024-04-25T11:38:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7fb7f6cfd640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e8008e20 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7fb7f4a99640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8046310 0x7fb7e80d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7fb7f529a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e80d5cb0 0x7fb7e80e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7fb7effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e8046760 0x7fb7e8050840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7fb7f529a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e80d5cb0 0x7fb7e80e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:38246/0 (socket says 172.21.15.40:38246) 2024-04-25T11:38:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7fb7f529a640 1 -- 172.21.15.40:0/1806908029 learned_addr learned my addr 172.21.15.40:0/1806908029 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7f8abb7fe640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8aa806c9f0 0x7f8aa806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.303+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40414e730 con 0x7fe40415a570 2024-04-25T11:38:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7fb7f529a640 1 -- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8046310 msgr2=0x7fb7e80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7fb7f529a640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8046310 0x7fb7e80d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7fb7f529a640 1 -- 172.21.15.40:0/1806908029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e8046760 msgr2=0x7fb7e8050840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7fb7f529a640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e8046760 0x7fb7e8050840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7fb7f529a640 1 -- 172.21.15.40:0/1806908029 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e8036fe0 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7f89bb920640 1 -- 172.21.15.40:0/2355841852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b406fe90 msgr2=0x7f89b4071fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7f89bb920640 1 --2- 172.21.15.40:0/2355841852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b406fe90 0x7f89b4071fa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009870 tx=0x7f89a402a2a0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7fb7f529a640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e80d5cb0 0x7fb7e80e8af0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00746a0 tx=0x7fb7f0063750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7f89bb920640 1 -- 172.21.15.40:0/2355841852 shutdown_connections 2024-04-25T11:38:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7f89bb920640 1 --2- 172.21.15.40:0/2355841852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b406fe90 0x7f89b4071fa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7f89bb920640 1 --2- 172.21.15.40:0/2355841852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b4159370 0x7f89b41626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7f89bb920640 1 -- 172.21.15.40:0/2355841852 >> 172.21.15.40:0/2355841852 conn(0x7f89b406b970 msgr2=0x7f89b406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7f89bb920640 1 -- 172.21.15.40:0/2355841852 shutdown_connections 2024-04-25T11:38:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.304+0000 7f89bb920640 1 -- 172.21.15.40:0/2355841852 wait complete. 2024-04-25T11:38:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7fe4017fa640 1 -- 172.21.15.40:0/2872932162 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3f003e940 con 0x7fe40415a570 2024-04-25T11:38:34.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7f0f91ffb640 1 -- 172.21.15.40:0/896732748 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f9803ccd0 con 0x7f0f9c06c490 2024-04-25T11:38:34.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9c04fd50 con 0x7f0f9c06c490 2024-04-25T11:38:34.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7fe4017fa640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe3e806c9c0 0x7fe3e806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7fb7ed7fa640 1 -- 172.21.15.40:0/1806908029 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00b3050 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7fb7ed7fa640 1 -- 172.21.15.40:0/1806908029 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f00b4040 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7fb7ed7fa640 1 -- 172.21.15.40:0/1806908029 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00adba0 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e80e7d30 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7fe4017fa640 1 -- 172.21.15.40:0/2872932162 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe3f00b56c0 con 0x7fe40415a570 2024-04-25T11:38:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7f0f91ffb640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f8006c8f0 0x7f0f8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.305+0000 7f0f91ffb640 1 -- 172.21.15.40:0/896732748 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0f980ac700 con 0x7f0f9c06c490 2024-04-25T11:38:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.306+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e80e82f0 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.306+0000 7f0fa0a6a640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f8006c8f0 0x7f0f8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.306+0000 7fe408a67640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe3e806c9c0 0x7fe3e806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.306+0000 7f8abb7fe640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8aa806c9f0 0x7f8aa806eeb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009a40 tx=0x7f8aac0091c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.307+0000 7f0fa0a6a640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f8006c8f0 0x7f0f8006edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c132b30 tx=0x7f0f8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.307+0000 7f529d605640 1 Processor -- start 2024-04-25T11:38:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.307+0000 7f89bb920640 1 Processor -- start 2024-04-25T11:38:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.307+0000 7f529d605640 1 -- start start 2024-04-25T11:38:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.307+0000 7f89bb920640 1 -- start start 2024-04-25T11:38:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.307+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e8005700 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.307+0000 7fe408a67640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe3e806c9c0 0x7fe3e806ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8005880 tx=0x7fe3f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89bb920640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b406fe90 0x7f89b40cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89bb920640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b4158f90 0x7f89b4160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89bb920640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b4159370 0x7f89b4160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89bb920640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4072ae0 con 0x7f89b4159370 2024-04-25T11:38:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89b96bc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b406fe90 0x7f89b40cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89bb920640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89b415fb80 con 0x7f89b4158f90 2024-04-25T11:38:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89b96bc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b406fe90 0x7f89b40cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:38256/0 (socket says 172.21.15.40:38256) 2024-04-25T11:38:34.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89b96bc640 1 -- 172.21.15.40:0/2859946855 learned_addr learned my addr 172.21.15.40:0/2859946855 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89b8ebb640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b4158f90 0x7f89b4160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f89b4072c60 con 0x7f89b406fe90 2024-04-25T11:38:34.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.308+0000 7f89b9ebd640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b4159370 0x7f89b4160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f529d605640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5298151440 0x7f52980c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f529d605640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806e4c0 0x7f52980cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f529d605640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5298157d80 con 0x7f529806c980 2024-04-25T11:38:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f89b96bc640 1 -- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b4158f90 msgr2=0x7f89b4160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f89b96bc640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b4158f90 0x7f89b4160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f89b96bc640 1 -- 172.21.15.40:0/2859946855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b4159370 msgr2=0x7f89b4160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f89b96bc640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b4159370 0x7f89b4160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f89b96bc640 1 -- 172.21.15.40:0/2859946855 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b41f4bc0 con 0x7f89b406fe90 2024-04-25T11:38:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f89b8ebb640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b4158f90 0x7f89b4160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f52977fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f529806c980 0x7f52981571a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44424/0 (socket says 172.21.15.40:44424) 2024-04-25T11:38:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f52977fe640 1 -- 172.21.15.40:0/2055337808 learned_addr learned my addr 172.21.15.40:0/2055337808 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f52967fc640 1 --2- 172.21.15.40:0/2055337808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806e4c0 0x7f52980cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f5296ffd640 1 --2- 172.21.15.40:0/2055337808 >> v2:172.21.15.40:3300/0 conn(0x7f5298151440 0x7f52980c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f529d605640 1 -- 172.21.15.40:0/2055337808 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5298158060 con 0x7f5298151440 2024-04-25T11:38:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f529d605640 1 -- 172.21.15.40:0/2055337808 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f52980ca740 con 0x7f529806e4c0 2024-04-25T11:38:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f89b96bc640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b406fe90 0x7f89b40cb4f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c430 tx=0x7f89b000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f8aa3fff640 1 -- 172.21.15.40:0/771952280 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8abc069a10 con 0x7f8abc072df0 2024-04-25T11:38:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f52967fc640 1 -- 172.21.15.40:0/2055337808 >> v2:172.21.15.40:3300/0 conn(0x7f5298151440 msgr2=0x7f52980c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f52967fc640 1 --2- 172.21.15.40:0/2055337808 >> v2:172.21.15.40:3300/0 conn(0x7f5298151440 0x7f52980c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f52967fc640 1 -- 172.21.15.40:0/2055337808 >> v1:172.21.15.40:6789/0 conn(0x7f529806c980 legacy=0x7f52981571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.309+0000 7f52967fc640 1 -- 172.21.15.40:0/2055337808 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52980caa20 con 0x7f529806e4c0 2024-04-25T11:38:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.310+0000 7fe4017fa640 1 -- 172.21.15.40:0/2872932162 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe40414e730 con 0x7fe40415a570 2024-04-25T11:38:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.310+0000 7f89a9ffb640 1 -- 172.21.15.40:0/2859946855 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0011790 con 0x7f89b406fe90 2024-04-25T11:38:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.310+0000 7f89a9ffb640 1 -- 172.21.15.40:0/2859946855 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b0017070 con 0x7f89b406fe90 2024-04-25T11:38:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.310+0000 7f89a9ffb640 1 -- 172.21.15.40:0/2859946855 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0011bb0 con 0x7f89b406fe90 2024-04-25T11:38:34.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.311+0000 7f52967fc640 1 --2- 172.21.15.40:0/2055337808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806e4c0 0x7f52980cb500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f528c00a8c0 tx=0x7f528c02b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9538b5352750da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.311+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b41f82b0 con 0x7f89b406fe90 2024-04-25T11:38:34.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.311+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b41f5120 con 0x7f89b406fe90 2024-04-25T11:38:34.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.312+0000 7fb7ed7fa640 1 -- 172.21.15.40:0/1806908029 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7f00c1030 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.312+0000 7fb7ed7fa640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb7d406c9f0 0x7fb7d406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.313+0000 7fb7ed7fa640 1 -- 172.21.15.40:0/1806908029 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb7f0128b90 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.313+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b404fd50 con 0x7f89b406fe90 2024-04-25T11:38:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.313+0000 7fb7f4a99640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb7d406c9f0 0x7fb7d406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.313+0000 7fb7f4a99640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb7d406c9f0 0x7fb7d406eeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4000f60 tx=0x7fb7e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.314+0000 7f5295ffb640 1 -- 172.21.15.40:0/2055337808 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c038070 con 0x7f529806e4c0 2024-04-25T11:38:34.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.315+0000 7f5295ffb640 1 -- 172.21.15.40:0/2055337808 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528c033640 con 0x7f529806e4c0 2024-04-25T11:38:34.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.315+0000 7f5295ffb640 1 -- 172.21.15.40:0/2055337808 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c038070 con 0x7f529806e4c0 2024-04-25T11:38:34.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.315+0000 7fb7ed7fa640 1 -- 172.21.15.40:0/1806908029 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7f00c8020 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.316+0000 7f529d605640 1 -- 172.21.15.40:0/2055337808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806e4c0 msgr2=0x7f52980cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.316+0000 7f529d605640 1 --2- 172.21.15.40:0/2055337808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806e4c0 0x7f52980cb500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f528c00a8c0 tx=0x7f528c02b330 comp rx=0 tx=0).stop 2024-04-25T11:38:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.316+0000 7f529d605640 1 -- 172.21.15.40:0/2055337808 shutdown_connections 2024-04-25T11:38:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.316+0000 7f529d605640 1 --2- 172.21.15.40:0/2055337808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806e4c0 0x7f52980cb500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.316+0000 7f0f91ffb640 1 -- 172.21.15.40:0/896732748 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f9807a980 con 0x7f0f9c06c490 2024-04-25T11:38:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.316+0000 7f529d605640 1 --2- 172.21.15.40:0/2055337808 >> v2:172.21.15.40:3300/0 conn(0x7f5298151440 0x7f52980c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.316+0000 7f529d605640 1 -- 172.21.15.40:0/2055337808 >> 172.21.15.40:0/2055337808 conn(0x7f529806d9a0 msgr2=0x7f529806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.316+0000 7f89a9ffb640 1 -- 172.21.15.40:0/2859946855 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89b0026430 con 0x7f89b406fe90 2024-04-25T11:38:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.316+0000 7f529d605640 1 -- 172.21.15.40:0/2055337808 shutdown_connections 2024-04-25T11:38:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.317+0000 7f529d605640 1 -- 172.21.15.40:0/2055337808 wait complete. 2024-04-25T11:38:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.317+0000 7f89a9ffb640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f899806c9c0 0x7f899806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.317+0000 7f89a9ffb640 1 -- 172.21.15.40:0/2859946855 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89b008c440 con 0x7f89b406fe90 2024-04-25T11:38:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.318+0000 7f89b8ebb640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f899806c9c0 0x7f899806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.318+0000 7f89b8ebb640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f899806c9c0 0x7f899806ee80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89b40c6340 tx=0x7f89ac00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.319+0000 7f529d605640 1 Processor -- start 2024-04-25T11:38:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.319+0000 7f529d605640 1 -- start start 2024-04-25T11:38:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.319+0000 7f89a9ffb640 1 -- 172.21.15.40:0/2859946855 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89b41f4bc0 con 0x7f89b406fe90 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.321+0000 7f529d605640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806c980 0x7f5298152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.321+0000 7f529d605640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529806e4c0 0x7f5298152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.321+0000 7f52967fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529806e4c0 0x7f5298152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.321+0000 7f52967fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529806e4c0 0x7f5298152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53494/0 (socket says 172.21.15.40:53494) 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.322+0000 7f52967fc640 1 -- 172.21.15.40:0/2313742091 learned_addr learned my addr 172.21.15.40:0/2313742091 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.321+0000 7f5296ffd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806c980 0x7f5298152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.322+0000 7f529d605640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5298151440 0x7f5298153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.322+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5298151820 con 0x7f529806e4c0 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.322+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52980cafe0 con 0x7f5298151440 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.322+0000 7f52977fe640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5298151440 0x7f5298153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.322+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f52981578b0 con 0x7f529806c980 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.323+0000 7f52967fc640 1 -- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806c980 msgr2=0x7f5298152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.323+0000 7f52967fc640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806c980 0x7f5298152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.323+0000 7f52967fc640 1 -- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5298151440 msgr2=0x7f5298153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.323+0000 7f52967fc640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5298151440 0x7f5298153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.323+0000 7f52967fc640 1 -- 172.21.15.40:0/2313742091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52981edd10 con 0x7f529806e4c0 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.324+0000 7f52977fe640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5298151440 0x7f5298153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.324+0000 7f5296ffd640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806c980 0x7f5298152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.324+0000 7f52967fc640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529806e4c0 0x7f5298152960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f528c00a130 tx=0x7f528c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.325+0000 7f527f7fe640 1 -- 172.21.15.40:0/2313742091 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c033e60 con 0x7f529806e4c0 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.325+0000 7f527f7fe640 1 -- 172.21.15.40:0/2313742091 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528c043070 con 0x7f529806e4c0 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.326+0000 7f527f7fe640 1 -- 172.21.15.40:0/2313742091 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c038070 con 0x7f529806e4c0 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.326+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529814e3f0 con 0x7f529806e4c0 2024-04-25T11:38:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.326+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5298151c20 con 0x7f529806e4c0 2024-04-25T11:38:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.328+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52980cbc30 con 0x7f529806e4c0 2024-04-25T11:38:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.340+0000 7f527f7fe640 1 -- 172.21.15.40:0/2313742091 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f528c007080 con 0x7f529806e4c0 2024-04-25T11:38:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.340+0000 7f527f7fe640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f528406c9f0 0x7f528406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.341+0000 7f527f7fe640 1 -- 172.21.15.40:0/2313742091 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f528c0ac8a0 con 0x7f529806e4c0 2024-04-25T11:38:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.341+0000 7f527f7fe640 1 -- 172.21.15.40:0/2313742091 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52980cbc30 con 0x7f529806e4c0 2024-04-25T11:38:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.346+0000 7f5296ffd640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f528406c9f0 0x7f528406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.366+0000 7f5296ffd640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f528406c9f0 0x7f528406eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5288009b90 tx=0x7f5288008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.686+0000 7fbbcec59640 1 -- 172.21.15.40:0/1199407017 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbba8005580 con 0x7fbbc8154ff0 2024-04-25T11:38:34.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.687+0000 7fbbc57fa640 1 -- 172.21.15.40:0/1199407017 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbbb405d010 con 0x7fbbc8154ff0 2024-04-25T11:38:34.674 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.690+0000 7fbbb2ffd640 1 -- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbbac06c9c0 msgr2=0x7fbbac06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.690+0000 7fbbb2ffd640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbbac06c9c0 0x7fbbac06ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc034000 tx=0x7fbbbc02c000 comp rx=0 tx=0).stop 2024-04-25T11:38:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.690+0000 7fbbb2ffd640 1 -- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc8154ff0 msgr2=0x7fbbc80c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.690+0000 7fbbb2ffd640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc8154ff0 0x7fbbc80c4310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400c540 tx=0x7fbbb400ca00 comp rx=0 tx=0).stop 2024-04-25T11:38:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.691+0000 7fbbb2ffd640 1 -- 172.21.15.40:0/1199407017 shutdown_connections 2024-04-25T11:38:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.691+0000 7fbbb2ffd640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbbac06c9c0 0x7fbbac06ee80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.691+0000 7fbbb2ffd640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbc815b9e0 0x7fbbc815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.691+0000 7fbbb2ffd640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc8154ff0 0x7fbbc80c4310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.691+0000 7fbbb2ffd640 1 --2- 172.21.15.40:0/1199407017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80b3fc0 0x7fbbc806f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.691+0000 7fbbb2ffd640 1 -- 172.21.15.40:0/1199407017 >> 172.21.15.40:0/1199407017 conn(0x7fbbc806c680 msgr2=0x7fbbc8150030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.691+0000 7fbbb2ffd640 1 -- 172.21.15.40:0/1199407017 shutdown_connections 2024-04-25T11:38:34.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.691+0000 7fbbb2ffd640 1 -- 172.21.15.40:0/1199407017 wait complete. 2024-04-25T11:38:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.706+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7ebc04fd50 con 0x7f7ebc151160 2024-04-25T11:38:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.706+0000 7f7eb97fa640 1 -- 172.21.15.40:0/2889937963 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7eac0593c0 con 0x7f7ebc151160 2024-04-25T11:38:34.694 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:34.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-04-25T11:38:34.696 DEBUG:teuthology.parallel:result is None 2024-04-25T11:38:34.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.709+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7ea806c8f0 msgr2=0x7f7ea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7ea806c8f0 0x7f7ea806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006fd0 tx=0x7f7eb0008040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc151160 msgr2=0x7f7ebc0725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc151160 0x7f7ebc0725e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00b7b0 tx=0x7f7eac00bc70 comp rx=0 tx=0).stop 2024-04-25T11:38:34.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 shutdown_connections 2024-04-25T11:38:34.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7ea806c8f0 0x7f7ea806edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc157b20 0x7f7ebc072b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ebc151160 0x7f7ebc0725e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 --2- 172.21.15.40:0/2889937963 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ebc0c7b00 0x7f7ebc1f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 >> 172.21.15.40:0/2889937963 conn(0x7f7ebc0c94b0 msgr2=0x7f7ebc06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 shutdown_connections 2024-04-25T11:38:34.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.710+0000 7f7ec3067640 1 -- 172.21.15.40:0/2889937963 wait complete. 2024-04-25T11:38:34.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2024-04-25T11:38:34.714 DEBUG:teuthology.parallel:result is None 2024-04-25T11:38:34.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.729+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f89b40c56a0 con 0x7f89b406fe90 2024-04-25T11:38:34.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.729+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8abc04fd50 con 0x7f8abc072df0 2024-04-25T11:38:34.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.729+0000 7f8aa3fff640 1 -- 172.21.15.40:0/771952280 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ab407b240 con 0x7f8abc072df0 2024-04-25T11:38:34.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.729+0000 7f89a9ffb640 1 -- 172.21.15.40:0/2859946855 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f89b005a850 con 0x7f89b406fe90 2024-04-25T11:38:34.717 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:34.717 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:34.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8aa806c9f0 msgr2=0x7f8aa806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8aa806c9f0 0x7f8aa806eeb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009a40 tx=0x7f8aac0091c0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc072df0 msgr2=0x7f8abc0cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc072df0 0x7f8abc0cbb80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4031000 tx=0x7f8ab4004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 shutdown_connections 2024-04-25T11:38:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8aa806c9f0 0x7f8aa806eeb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8abc0c8940 0x7f8abc161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f899806c9c0 msgr2=0x7f899806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8abc072df0 0x7f8abc0cbb80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 --2- 172.21.15.40:0/771952280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8abc072a10 0x7f8abc150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 >> 172.21.15.40:0/771952280 conn(0x7f8abc0cd170 msgr2=0x7f8abc0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f89bb920640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f899806c9c0 0x7f899806ee80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89b40c6340 tx=0x7f89ac00b040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b406fe90 msgr2=0x7f89b40cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.731+0000 7f89bb920640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b406fe90 0x7f89b40cb4f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c430 tx=0x7f89b000c8f0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 shutdown_connections 2024-04-25T11:38:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f89bb920640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f899806c9c0 0x7f899806ee80 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89b40c6340 tx=0x7f89ac00b040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f89bb920640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b4159370 0x7f89b4160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f89bb920640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b4158f90 0x7f89b4160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f89bb920640 1 --2- 172.21.15.40:0/2859946855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89b406fe90 0x7f89b40cb4f0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c430 tx=0x7f89b000c8f0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 >> 172.21.15.40:0/2859946855 conn(0x7f89b406b970 msgr2=0x7f89b40c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 shutdown_connections 2024-04-25T11:38:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 shutdown_connections 2024-04-25T11:38:34.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f8ac1eef640 1 -- 172.21.15.40:0/771952280 wait complete. 2024-04-25T11:38:34.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.732+0000 7f89bb920640 1 -- 172.21.15.40:0/2859946855 wait complete. 2024-04-25T11:38:34.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2024-04-25T11:38:34.731 DEBUG:teuthology.parallel:result is None 2024-04-25T11:38:34.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2024-04-25T11:38:34.732 DEBUG:teuthology.parallel:result is None 2024-04-25T11:38:34.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.750+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe40404fce0 con 0x7fe40415a570 2024-04-25T11:38:34.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.751+0000 7fe4017fa640 1 -- 172.21.15.40:0/2872932162 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe3f0045070 con 0x7fe40415a570 2024-04-25T11:38:34.738 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:34.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe3e806c9c0 msgr2=0x7fe3e806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe3e806c9c0 0x7fe3e806ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8005880 tx=0x7fe3f8009040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40415a570 msgr2=0x7fe404144680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40415a570 0x7fe404144680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000df10 tx=0x7fe3f00046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 shutdown_connections 2024-04-25T11:38:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe3e806c9c0 0x7fe3e806ee80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4041efcd0 0x7fe4041edc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe40415a570 0x7fe404144680 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 --2- 172.21.15.40:0/2872932162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe404151180 0x7fe404146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 >> 172.21.15.40:0/2872932162 conn(0x7fe4040cc940 msgr2=0x7fe4040c5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 shutdown_connections 2024-04-25T11:38:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.754+0000 7fe409a69640 1 -- 172.21.15.40:0/2872932162 wait complete. 2024-04-25T11:38:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.755+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0f9c072190 con 0x7f0f9c06c490 2024-04-25T11:38:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.755+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb7e80e7190 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.755+0000 7f0f91ffb640 1 -- 172.21.15.40:0/896732748 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0f9807a280 con 0x7f0f9c06c490 2024-04-25T11:38:34.744 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.756+0000 7fb7ed7fa640 1 -- 172.21.15.40:0/1806908029 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb7f0128070 con 0x7fb7e80d5cb0 2024-04-25T11:38:34.744 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f8006c8f0 msgr2=0x7f0f8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f8006c8f0 0x7f0f8006edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c132b30 tx=0x7f0f8c008040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9c06c490 msgr2=0x7f0f9c13b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9c06c490 0x7f0f9c13b280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f9802a7d0 tx=0x7f0f980046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 shutdown_connections 2024-04-25T11:38:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f8006c8f0 0x7f0f8006edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f9c151040 0x7f0f9c132c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f9c072900 0x7f0f9c165b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 --2- 172.21.15.40:0/896732748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9c06c490 0x7f0f9c13b280 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 >> 172.21.15.40:0/896732748 conn(0x7f0f9c06b970 msgr2=0x7f0f9c14d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 shutdown_connections 2024-04-25T11:38:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb7d406c9f0 msgr2=0x7fb7d406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb7d406c9f0 0x7fb7d406eeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4000f60 tx=0x7fb7e4009040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7f0fa34cf640 1 -- 172.21.15.40:0/896732748 wait complete. 2024-04-25T11:38:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e80d5cb0 msgr2=0x7fb7e80e8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e80d5cb0 0x7fb7e80e8af0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00746a0 tx=0x7fb7f0063750 comp rx=0 tx=0).stop 2024-04-25T11:38:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 shutdown_connections 2024-04-25T11:38:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb7d406c9f0 0x7fb7d406eeb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7e80d5cb0 0x7fb7e80e8af0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e8046760 0x7fb7e8050840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 --2- 172.21.15.40:0/1806908029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8046310 0x7fb7e80d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 >> 172.21.15.40:0/1806908029 conn(0x7fb7e8051280 msgr2=0x7fb7e80d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 shutdown_connections 2024-04-25T11:38:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.759+0000 7fb7f6cfd640 1 -- 172.21.15.40:0/1806908029 wait complete. 2024-04-25T11:38:34.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2024-04-25T11:38:34.756 DEBUG:teuthology.parallel:result is None 2024-04-25T11:38:34.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-04-25T11:38:34.759 DEBUG:teuthology.parallel:result is None 2024-04-25T11:38:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.775+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f529804fce0 con 0x7f529806e4c0 2024-04-25T11:38:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.775+0000 7f527f7fe640 1 -- 172.21.15.40:0/2313742091 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f528c04c060 con 0x7f529806e4c0 2024-04-25T11:38:34.763 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-25T11:38:34.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-04-25T11:38:34.764 DEBUG:teuthology.parallel:result is None 2024-04-25T11:38:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.777+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f528406c9f0 msgr2=0x7f528406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.777+0000 7f529d605640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f528406c9f0 0x7f528406eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5288009b90 tx=0x7f5288008040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529806e4c0 msgr2=0x7f5298152960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529806e4c0 0x7f5298152960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f528c00a130 tx=0x7f528c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 shutdown_connections 2024-04-25T11:38:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f528406c9f0 0x7f528406eeb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5298151440 0x7f5298153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529806e4c0 0x7f5298152960 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 --2- 172.21.15.40:0/2313742091 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f529806c980 0x7f5298152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 >> 172.21.15.40:0/2313742091 conn(0x7f529806d9a0 msgr2=0x7f52980c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 shutdown_connections 2024-04-25T11:38:34.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.778+0000 7f529d605640 1 -- 172.21.15.40:0/2313742091 wait complete. 2024-04-25T11:38:34.776 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-04-25T11:38:34.776 DEBUG:teuthology.parallel:result is None 2024-04-25T11:38:34.776 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:38:34.776 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:38:34.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.930+0000 7fcdf5a10640 1 Processor -- start 2024-04-25T11:38:34.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.930+0000 7fcdf5a10640 1 -- start start 2024-04-25T11:38:34.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdf5a10640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdf00c97f0 0x7fcdf00c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdf5a10640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdf01553f0 con 0x7fcdf0155010 2024-04-25T11:38:34.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdf5a10640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdf00ce7c0 con 0x7fcdf006ca80 2024-04-25T11:38:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdf5a10640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf006eab0 con 0x7fcdf00c97f0 2024-04-25T11:38:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdee7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdf00c97f0 0x7fcdf00c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdeeffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcdf0155010 0x7fcdf015eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44432/0 (socket says 172.21.15.40:44432) 2024-04-25T11:38:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdeeffd640 1 -- 172.21.15.40:0/3320958681 learned_addr learned my addr 172.21.15.40:0/3320958681 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdee7fc640 1 -- 172.21.15.40:0/3320958681 >> v1:172.21.15.62:6789/0 conn(0x7fcdf006ca80 legacy=0x7fcdf0165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdee7fc640 1 -- 172.21.15.40:0/3320958681 >> v1:172.21.15.40:6789/0 conn(0x7fcdf0155010 legacy=0x7fcdf015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdee7fc640 1 -- 172.21.15.40:0/3320958681 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf00ceaa0 con 0x7fcdf00c97f0 2024-04-25T11:38:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.931+0000 7fcdedffb640 1 -- 172.21.15.40:0/3320958681 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4285880166 0 0) 0x7fcdf01553f0 con 0x7fcdf0155010 2024-04-25T11:38:34.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.932+0000 7fcdee7fc640 1 --2- 172.21.15.40:0/3320958681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 0x7fcdf00c8c50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80097a0 tx=0x7fcdd802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209f1b5358fa03c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.932+0000 7fcdedffb640 1 -- 172.21.15.40:0/3320958681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8037070 con 0x7fcdf00c97f0 2024-04-25T11:38:34.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.932+0000 7fcdedffb640 1 -- 172.21.15.40:0/3320958681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd8032440 con 0x7fcdf00c97f0 2024-04-25T11:38:34.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.932+0000 7fcdedffb640 1 -- 172.21.15.40:0/3320958681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8032810 con 0x7fcdf00c97f0 2024-04-25T11:38:34.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.933+0000 7fcdf5a10640 1 -- 172.21.15.40:0/3320958681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 msgr2=0x7fcdf00c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.933+0000 7fcdf5a10640 1 --2- 172.21.15.40:0/3320958681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 0x7fcdf00c8c50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80097a0 tx=0x7fcdd802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.933+0000 7fcdf5a10640 1 -- 172.21.15.40:0/3320958681 shutdown_connections 2024-04-25T11:38:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.933+0000 7fcdf5a10640 1 --2- 172.21.15.40:0/3320958681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 0x7fcdf00c8c50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.933+0000 7fcdf5a10640 1 -- 172.21.15.40:0/3320958681 >> 172.21.15.40:0/3320958681 conn(0x7fcdf006b970 msgr2=0x7fcdf006f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.933+0000 7fcdf5a10640 1 -- 172.21.15.40:0/3320958681 shutdown_connections 2024-04-25T11:38:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.933+0000 7fcdf5a10640 1 -- 172.21.15.40:0/3320958681 wait complete. 2024-04-25T11:38:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.934+0000 7fcdf5a10640 1 Processor -- start 2024-04-25T11:38:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.934+0000 7fcdf5a10640 1 -- start start 2024-04-25T11:38:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdf5a10640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdf006ca80 0x7fcdf015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdf5a10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 0x7fcdf01f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdf5a10640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdf0155010 0x7fcdf0147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdf5a10640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf00ced80 con 0x7fcdf00c97f0 2024-04-25T11:38:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdf5a10640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf00c9300 con 0x7fcdf0155010 2024-04-25T11:38:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdf5a10640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf015e5f0 con 0x7fcdf006ca80 2024-04-25T11:38:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 0x7fcdf01f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 0x7fcdf01f1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53524/0 (socket says 172.21.15.40:53524) 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdee7fc640 1 -- 172.21.15.40:0/1855770161 learned_addr learned my addr 172.21.15.40:0/1855770161 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdeeffd640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdf006ca80 0x7fcdf015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdef7fe640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdf0155010 0x7fcdf0147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdee7fc640 1 -- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdf006ca80 msgr2=0x7fcdf015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdee7fc640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdf006ca80 0x7fcdf015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdee7fc640 1 -- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdf0155010 msgr2=0x7fcdf0147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdee7fc640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdf0155010 0x7fcdf0147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdee7fc640 1 -- 172.21.15.40:0/1855770161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf01f2490 con 0x7fcdf00c97f0 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdee7fc640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 0x7fcdf01f1f50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80097a0 tx=0x7fcdd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcde7fff640 1 -- 172.21.15.40:0/1855770161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8037050 con 0x7fcdf00c97f0 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcde7fff640 1 -- 172.21.15.40:0/1855770161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd8038040 con 0x7fcdf00c97f0 2024-04-25T11:38:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcde7fff640 1 -- 172.21.15.40:0/1855770161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8004490 con 0x7fcdf00c97f0 2024-04-25T11:38:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.935+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf015dae0 con 0x7fcdf00c97f0 2024-04-25T11:38:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.936+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf01f0cd0 con 0x7fcdf00c97f0 2024-04-25T11:38:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.937+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf01f0ae0 con 0x7fcdf00c97f0 2024-04-25T11:38:34.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.937+0000 7fcde7fff640 1 -- 172.21.15.40:0/1855770161 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdd8030900 con 0x7fcdf00c97f0 2024-04-25T11:38:34.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.937+0000 7fcde7fff640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcdd006c8f0 0x7fcdd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.938+0000 7fcde7fff640 1 -- 172.21.15.40:0/1855770161 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcdd80ac7a0 con 0x7fcdf00c97f0 2024-04-25T11:38:34.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.938+0000 7fcdeeffd640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcdd006c8f0 0x7fcdd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.938+0000 7fcdeeffd640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcdd006c8f0 0x7fcdd006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcde8007f70 tx=0x7fcde8007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:34.942+0000 7fcde7fff640 1 -- 172.21.15.40:0/1855770161 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcdd807a610 con 0x7fcdf00c97f0 2024-04-25T11:38:35.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.127+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcdf0068bb0 con 0x7fcdd006c8f0 2024-04-25T11:38:35.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.134+0000 7fcde7fff640 1 -- 172.21.15.40:0/1855770161 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251912 (secure 0 0 0) 0x7fcdf0068bb0 con 0x7fcdd006c8f0 2024-04-25T11:38:35.124 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:35.125 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:38:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcdd006c8f0 msgr2=0x7fcdd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcdd006c8f0 0x7fcdd006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcde8007f70 tx=0x7fcde8007f00 comp rx=0 tx=0).stop 2024-04-25T11:38:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 msgr2=0x7fcdf01f1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 0x7fcdf01f1f50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80097a0 tx=0x7fcdd80046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 shutdown_connections 2024-04-25T11:38:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcdd006c8f0 0x7fcdd006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdf0155010 0x7fcdf0147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf00c97f0 0x7fcdf01f1f50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 --2- 172.21.15.40:0/1855770161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdf006ca80 0x7fcdf015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 >> 172.21.15.40:0/1855770161 conn(0x7fcdf006b970 msgr2=0x7fcdf01e64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 shutdown_connections 2024-04-25T11:38:35.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.137+0000 7fcdf5a10640 1 -- 172.21.15.40:0/1855770161 wait complete. 2024-04-25T11:38:35.135 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T11:38:34.953029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217920,"kb_used_data":1792,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756800,"statfs":{"total":773094113280,"available":772870963200,"internally_reserved":0,"allocated":1835008,"data_stored":1753028,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.297728"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762879+0000","last_change":"2024-04-25T11:38:25.677572+0000","last_active":"2024-04-25T11:38:26.762879+0000","last_peered":"2024-04-25T11:38:26.762879+0000","last_clean":"2024-04-25T11:38:26.762879+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:38:26.762879+0000","last_undegraded":"2024-04-25T11:38:26.762879+0000","last_fullsized":"2024-04-25T11:38:26.762879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673367+0000","last_change":"2024-04-25T11:38:25.677437+0000","last_active":"2024-04-25T11:38:26.673367+0000","last_peered":"2024-04-25T11:38:26.673367+0000","last_clean":"2024-04-25T11:38:26.673367+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:38:26.673367+0000","last_undegraded":"2024-04-25T11:38:26.673367+0000","last_fullsized":"2024-04-25T11:38:26.673367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761847+0000","last_change":"2024-04-25T11:38:25.676553+0000","last_active":"2024-04-25T11:38:26.761847+0000","last_peered":"2024-04-25T11:38:26.761847+0000","last_clean":"2024-04-25T11:38:26.761847+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:38:26.761847+0000","last_undegraded":"2024-04-25T11:38:26.761847+0000","last_fullsized":"2024-04-25T11:38:26.761847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762862+0000","last_change":"2024-04-25T11:38:25.676518+0000","last_active":"2024-04-25T11:38:26.762862+0000","last_peered":"2024-04-25T11:38:26.762862+0000","last_clean":"2024-04-25T11:38:26.762862+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:38:26.762862+0000","last_undegraded":"2024-04-25T11:38:26.762862+0000","last_fullsized":"2024-04-25T11:38:26.762862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762062+0000","last_change":"2024-04-25T11:38:25.678524+0000","last_active":"2024-04-25T11:38:26.762062+0000","last_peered":"2024-04-25T11:38:26.762062+0000","last_clean":"2024-04-25T11:38:26.762062+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:38:26.762062+0000","last_undegraded":"2024-04-25T11:38:26.762062+0000","last_fullsized":"2024-04-25T11:38:26.762062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761774+0000","last_change":"2024-04-25T11:38:25.683983+0000","last_active":"2024-04-25T11:38:26.761774+0000","last_peered":"2024-04-25T11:38:26.761774+0000","last_clean":"2024-04-25T11:38:26.761774+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:38:26.761774+0000","last_undegraded":"2024-04-25T11:38:26.761774+0000","last_fullsized":"2024-04-25T11:38:26.761774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761720+0000","last_change":"2024-04-25T11:38:25.682904+0000","last_active":"2024-04-25T11:38:26.761720+0000","last_peered":"2024-04-25T11:38:26.761720+0000","last_clean":"2024-04-25T11:38:26.761720+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:38:26.761720+0000","last_undegraded":"2024-04-25T11:38:26.761720+0000","last_fullsized":"2024-04-25T11:38:26.761720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761274+0000","last_change":"2024-04-25T11:38:25.670312+0000","last_active":"2024-04-25T11:38:26.761274+0000","last_peered":"2024-04-25T11:38:26.761274+0000","last_clean":"2024-04-25T11:38:26.761274+0000","last_became_active":"2024-04-25T11:38:25.669988+0000","last_became_peered":"2024-04-25T11:38:25.669988+0000","last_unstale":"2024-04-25T11:38:26.761274+0000","last_undegraded":"2024-04-25T11:38:26.761274+0000","last_fullsized":"2024-04-25T11:38:26.761274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762939+0000","last_change":"2024-04-25T11:38:25.678459+0000","last_active":"2024-04-25T11:38:26.762939+0000","last_peered":"2024-04-25T11:38:26.762939+0000","last_clean":"2024-04-25T11:38:26.762939+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:38:26.762939+0000","last_undegraded":"2024-04-25T11:38:26.762939+0000","last_fullsized":"2024-04-25T11:38:26.762939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674629+0000","last_change":"2024-04-25T11:38:25.675561+0000","last_active":"2024-04-25T11:38:26.674629+0000","last_peered":"2024-04-25T11:38:26.674629+0000","last_clean":"2024-04-25T11:38:26.674629+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:38:26.674629+0000","last_undegraded":"2024-04-25T11:38:26.674629+0000","last_fullsized":"2024-04-25T11:38:26.674629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674251+0000","last_change":"2024-04-25T11:38:25.671113+0000","last_active":"2024-04-25T11:38:26.674251+0000","last_peered":"2024-04-25T11:38:26.674251+0000","last_clean":"2024-04-25T11:38:26.674251+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:38:26.674251+0000","last_undegraded":"2024-04-25T11:38:26.674251+0000","last_fullsized":"2024-04-25T11:38:26.674251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762821+0000","last_change":"2024-04-25T11:38:25.665690+0000","last_active":"2024-04-25T11:38:26.762821+0000","last_peered":"2024-04-25T11:38:26.762821+0000","last_clean":"2024-04-25T11:38:26.762821+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:38:26.762821+0000","last_undegraded":"2024-04-25T11:38:26.762821+0000","last_fullsized":"2024-04-25T11:38:26.762821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:25.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761152+0000","last_change":"2024-04-25T11:38:25.666858+0000","last_active":"2024-04-25T11:38:26.761152+0000","last_peered":"2024-04-25T11:38:26.761152+0000","last_clean":"2024-04-25T11:38:26.761152+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:38:26.761152+0000","last_undegraded":"2024-04-25T11:38:26.761152+0000","last_fullsized":"2024-04-25T11:38:26.761152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761293+0000","last_change":"2024-04-25T11:38:25.677561+0000","last_active":"2024-04-25T11:38:26.761293+0000","last_peered":"2024-04-25T11:38:26.761293+0000","last_clean":"2024-04-25T11:38:26.761293+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:38:26.761293+0000","last_undegraded":"2024-04-25T11:38:26.761293+0000","last_fullsized":"2024-04-25T11:38:26.761293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673120+0000","last_change":"2024-04-25T11:38:25.672129+0000","last_active":"2024-04-25T11:38:26.673120+0000","last_peered":"2024-04-25T11:38:26.673120+0000","last_clean":"2024-04-25T11:38:26.673120+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:38:26.673120+0000","last_undegraded":"2024-04-25T11:38:26.673120+0000","last_fullsized":"2024-04-25T11:38:26.673120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761886+0000","last_change":"2024-04-25T11:38:25.682547+0000","last_active":"2024-04-25T11:38:26.761886+0000","last_peered":"2024-04-25T11:38:26.761886+0000","last_clean":"2024-04-25T11:38:26.761886+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:38:26.761886+0000","last_undegraded":"2024-04-25T11:38:26.761886+0000","last_fullsized":"2024-04-25T11:38:26.761886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763750+0000","last_change":"2024-04-25T11:38:25.683798+0000","last_active":"2024-04-25T11:38:26.763750+0000","last_peered":"2024-04-25T11:38:26.763750+0000","last_clean":"2024-04-25T11:38:26.763750+0000","last_became_active":"2024-04-25T11:38:25.682590+0000","last_became_peered":"2024-04-25T11:38:25.682590+0000","last_unstale":"2024-04-25T11:38:26.763750+0000","last_undegraded":"2024-04-25T11:38:26.763750+0000","last_fullsized":"2024-04-25T11:38:26.763750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673869+0000","last_change":"2024-04-25T11:38:25.669650+0000","last_active":"2024-04-25T11:38:26.673869+0000","last_peered":"2024-04-25T11:38:26.673869+0000","last_clean":"2024-04-25T11:38:26.673869+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:38:26.673869+0000","last_undegraded":"2024-04-25T11:38:26.673869+0000","last_fullsized":"2024-04-25T11:38:26.673869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762614+0000","last_change":"2024-04-25T11:38:25.678238+0000","last_active":"2024-04-25T11:38:26.762614+0000","last_peered":"2024-04-25T11:38:26.762614+0000","last_clean":"2024-04-25T11:38:26.762614+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:38:26.762614+0000","last_undegraded":"2024-04-25T11:38:26.762614+0000","last_fullsized":"2024-04-25T11:38:26.762614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674167+0000","last_change":"2024-04-25T11:38:25.666736+0000","last_active":"2024-04-25T11:38:26.674167+0000","last_peered":"2024-04-25T11:38:26.674167+0000","last_clean":"2024-04-25T11:38:26.674167+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:38:26.674167+0000","last_undegraded":"2024-04-25T11:38:26.674167+0000","last_fullsized":"2024-04-25T11:38:26.674167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761661+0000","last_change":"2024-04-25T11:38:25.680587+0000","last_active":"2024-04-25T11:38:26.761661+0000","last_peered":"2024-04-25T11:38:26.761661+0000","last_clean":"2024-04-25T11:38:26.761661+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:38:26.761661+0000","last_undegraded":"2024-04-25T11:38:26.761661+0000","last_fullsized":"2024-04-25T11:38:26.761661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762157+0000","last_change":"2024-04-25T11:38:25.680464+0000","last_active":"2024-04-25T11:38:26.762157+0000","last_peered":"2024-04-25T11:38:26.762157+0000","last_clean":"2024-04-25T11:38:26.762157+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:38:26.762157+0000","last_undegraded":"2024-04-25T11:38:26.762157+0000","last_fullsized":"2024-04-25T11:38:26.762157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761201+0000","last_change":"2024-04-25T11:38:25.672189+0000","last_active":"2024-04-25T11:38:26.761201+0000","last_peered":"2024-04-25T11:38:26.761201+0000","last_clean":"2024-04-25T11:38:26.761201+0000","last_became_active":"2024-04-25T11:38:25.671920+0000","last_became_peered":"2024-04-25T11:38:25.671920+0000","last_unstale":"2024-04-25T11:38:26.761201+0000","last_undegraded":"2024-04-25T11:38:26.761201+0000","last_fullsized":"2024-04-25T11:38:26.761201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762072+0000","last_change":"2024-04-25T11:38:25.674108+0000","last_active":"2024-04-25T11:38:26.762072+0000","last_peered":"2024-04-25T11:38:26.762072+0000","last_clean":"2024-04-25T11:38:26.762072+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:38:26.762072+0000","last_undegraded":"2024-04-25T11:38:26.762072+0000","last_fullsized":"2024-04-25T11:38:26.762072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761259+0000","last_change":"2024-04-25T11:38:25.672215+0000","last_active":"2024-04-25T11:38:26.761259+0000","last_peered":"2024-04-25T11:38:26.761259+0000","last_clean":"2024-04-25T11:38:26.761259+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:38:26.761259+0000","last_undegraded":"2024-04-25T11:38:26.761259+0000","last_fullsized":"2024-04-25T11:38:26.761259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674666+0000","last_change":"2024-04-25T11:38:25.679772+0000","last_active":"2024-04-25T11:38:26.674666+0000","last_peered":"2024-04-25T11:38:26.674666+0000","last_clean":"2024-04-25T11:38:26.674666+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:38:26.674666+0000","last_undegraded":"2024-04-25T11:38:26.674666+0000","last_fullsized":"2024-04-25T11:38:26.674666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673164+0000","last_change":"2024-04-25T11:38:25.665626+0000","last_active":"2024-04-25T11:38:26.673164+0000","last_peered":"2024-04-25T11:38:26.673164+0000","last_clean":"2024-04-25T11:38:26.673164+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:38:26.673164+0000","last_undegraded":"2024-04-25T11:38:26.673164+0000","last_fullsized":"2024-04-25T11:38:26.673164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761385+0000","last_change":"2024-04-25T11:38:25.672935+0000","last_active":"2024-04-25T11:38:26.761385+0000","last_peered":"2024-04-25T11:38:26.761385+0000","last_clean":"2024-04-25T11:38:26.761385+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:38:26.761385+0000","last_undegraded":"2024-04-25T11:38:26.761385+0000","last_fullsized":"2024-04-25T11:38:26.761385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761842+0000","last_change":"2024-04-25T11:38:25.679486+0000","last_active":"2024-04-25T11:38:26.761842+0000","last_peered":"2024-04-25T11:38:26.761842+0000","last_clean":"2024-04-25T11:38:26.761842+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:38:26.761842+0000","last_undegraded":"2024-04-25T11:38:26.761842+0000","last_fullsized":"2024-04-25T11:38:26.761842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762406+0000","last_change":"2024-04-25T11:38:25.679437+0000","last_active":"2024-04-25T11:38:26.762406+0000","last_peered":"2024-04-25T11:38:26.762406+0000","last_clean":"2024-04-25T11:38:26.762406+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:38:26.762406+0000","last_undegraded":"2024-04-25T11:38:26.762406+0000","last_fullsized":"2024-04-25T11:38:26.762406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673702+0000","last_change":"2024-04-25T11:38:25.670384+0000","last_active":"2024-04-25T11:38:26.673702+0000","last_peered":"2024-04-25T11:38:26.673702+0000","last_clean":"2024-04-25T11:38:26.673702+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:38:26.673702+0000","last_undegraded":"2024-04-25T11:38:26.673702+0000","last_fullsized":"2024-04-25T11:38:26.673702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674250+0000","last_change":"2024-04-25T11:38:25.680252+0000","last_active":"2024-04-25T11:38:26.674250+0000","last_peered":"2024-04-25T11:38:26.674250+0000","last_clean":"2024-04-25T11:38:26.674250+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:38:26.674250+0000","last_undegraded":"2024-04-25T11:38:26.674250+0000","last_fullsized":"2024-04-25T11:38:26.674250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762291+0000","last_change":"2024-04-25T11:38:25.669542+0000","last_active":"2024-04-25T11:38:26.762291+0000","last_peered":"2024-04-25T11:38:26.762291+0000","last_clean":"2024-04-25T11:38:26.762291+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:38:26.762291+0000","last_undegraded":"2024-04-25T11:38:26.762291+0000","last_fullsized":"2024-04-25T11:38:26.762291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:51.929330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.590981+0000","last_change":"2024-04-25T11:38:24.666197+0000","last_active":"2024-04-25T11:38:28.590981+0000","last_peered":"2024-04-25T11:38:28.590981+0000","last_clean":"2024-04-25T11:38:28.590981+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:38:28.590981+0000","last_undegraded":"2024-04-25T11:38:28.590981+0000","last_fullsized":"2024-04-25T11:38:28.590981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.198972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761974+0000","last_change":"2024-04-25T11:38:25.682931+0000","last_active":"2024-04-25T11:38:26.761974+0000","last_peered":"2024-04-25T11:38:26.761974+0000","last_clean":"2024-04-25T11:38:26.761974+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:38:26.761974+0000","last_undegraded":"2024-04-25T11:38:26.761974+0000","last_fullsized":"2024-04-25T11:38:26.761974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.590777+0000","last_change":"2024-04-25T11:38:24.660888+0000","last_active":"2024-04-25T11:38:28.590777+0000","last_peered":"2024-04-25T11:38:28.590777+0000","last_clean":"2024-04-25T11:38:28.590777+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:38:28.590777+0000","last_undegraded":"2024-04-25T11:38:28.590777+0000","last_fullsized":"2024-04-25T11:38:28.590777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761513+0000","last_change":"2024-04-25T11:38:24.657815+0000","last_active":"2024-04-25T11:38:26.761513+0000","last_peered":"2024-04-25T11:38:26.761513+0000","last_clean":"2024-04-25T11:38:26.761513+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:38:26.761513+0000","last_undegraded":"2024-04-25T11:38:26.761513+0000","last_fullsized":"2024-04-25T11:38:26.761513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761437+0000","last_change":"2024-04-25T11:38:25.680969+0000","last_active":"2024-04-25T11:38:26.761437+0000","last_peered":"2024-04-25T11:38:26.761437+0000","last_clean":"2024-04-25T11:38:26.761437+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:38:26.761437+0000","last_undegraded":"2024-04-25T11:38:26.761437+0000","last_fullsized":"2024-04-25T11:38:26.761437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762182+0000","last_change":"2024-04-25T11:38:24.668899+0000","last_active":"2024-04-25T11:38:26.762182+0000","last_peered":"2024-04-25T11:38:26.762182+0000","last_clean":"2024-04-25T11:38:26.762182+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:38:26.762182+0000","last_undegraded":"2024-04-25T11:38:26.762182+0000","last_fullsized":"2024-04-25T11:38:26.762182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672975+0000","last_change":"2024-04-25T11:38:25.666511+0000","last_active":"2024-04-25T11:38:26.672975+0000","last_peered":"2024-04-25T11:38:26.672975+0000","last_clean":"2024-04-25T11:38:26.672975+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:38:26.672975+0000","last_undegraded":"2024-04-25T11:38:26.672975+0000","last_fullsized":"2024-04-25T11:38:26.672975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674478+0000","last_change":"2024-04-25T11:38:24.666735+0000","last_active":"2024-04-25T11:38:26.674478+0000","last_peered":"2024-04-25T11:38:26.674478+0000","last_clean":"2024-04-25T11:38:26.674478+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:38:26.674478+0000","last_undegraded":"2024-04-25T11:38:26.674478+0000","last_fullsized":"2024-04-25T11:38:26.674478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763586+0000","last_change":"2024-04-25T11:38:25.680065+0000","last_active":"2024-04-25T11:38:26.763586+0000","last_peered":"2024-04-25T11:38:26.763586+0000","last_clean":"2024-04-25T11:38:26.763586+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:38:26.763586+0000","last_undegraded":"2024-04-25T11:38:26.763586+0000","last_fullsized":"2024-04-25T11:38:26.763586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763667+0000","last_change":"2024-04-25T11:38:24.671566+0000","last_active":"2024-04-25T11:38:26.763667+0000","last_peered":"2024-04-25T11:38:26.763667+0000","last_clean":"2024-04-25T11:38:26.763667+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:38:26.763667+0000","last_undegraded":"2024-04-25T11:38:26.763667+0000","last_fullsized":"2024-04-25T11:38:26.763667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761910+0000","last_change":"2024-04-25T11:38:25.678075+0000","last_active":"2024-04-25T11:38:26.761910+0000","last_peered":"2024-04-25T11:38:26.761910+0000","last_clean":"2024-04-25T11:38:26.761910+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:38:26.761910+0000","last_undegraded":"2024-04-25T11:38:26.761910+0000","last_fullsized":"2024-04-25T11:38:26.761910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.678792+0000","last_change":"2024-04-25T11:38:24.667282+0000","last_active":"2024-04-25T11:38:30.678792+0000","last_peered":"2024-04-25T11:38:30.678792+0000","last_clean":"2024-04-25T11:38:30.678792+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:38:30.678792+0000","last_undegraded":"2024-04-25T11:38:30.678792+0000","last_fullsized":"2024-04-25T11:38:30.678792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762833+0000","last_change":"2024-04-25T11:38:25.680417+0000","last_active":"2024-04-25T11:38:26.762833+0000","last_peered":"2024-04-25T11:38:26.762833+0000","last_clean":"2024-04-25T11:38:26.762833+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:38:26.762833+0000","last_undegraded":"2024-04-25T11:38:26.762833+0000","last_fullsized":"2024-04-25T11:38:26.762833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.680566+0000","last_change":"2024-04-25T11:38:24.662562+0000","last_active":"2024-04-25T11:38:30.680566+0000","last_peered":"2024-04-25T11:38:30.680566+0000","last_clean":"2024-04-25T11:38:30.680566+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:38:30.680566+0000","last_undegraded":"2024-04-25T11:38:30.680566+0000","last_fullsized":"2024-04-25T11:38:30.680566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762135+0000","last_change":"2024-04-25T11:38:25.677518+0000","last_active":"2024-04-25T11:38:26.762135+0000","last_peered":"2024-04-25T11:38:26.762135+0000","last_clean":"2024-04-25T11:38:26.762135+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:38:26.762135+0000","last_undegraded":"2024-04-25T11:38:26.762135+0000","last_fullsized":"2024-04-25T11:38:26.762135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679081+0000","last_change":"2024-04-25T11:38:24.658960+0000","last_active":"2024-04-25T11:38:30.679081+0000","last_peered":"2024-04-25T11:38:30.679081+0000","last_clean":"2024-04-25T11:38:30.679081+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:38:30.679081+0000","last_undegraded":"2024-04-25T11:38:30.679081+0000","last_fullsized":"2024-04-25T11:38:30.679081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761438+0000","last_change":"2024-04-25T11:38:25.681355+0000","last_active":"2024-04-25T11:38:26.761438+0000","last_peered":"2024-04-25T11:38:26.761438+0000","last_clean":"2024-04-25T11:38:26.761438+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:38:26.761438+0000","last_undegraded":"2024-04-25T11:38:26.761438+0000","last_fullsized":"2024-04-25T11:38:26.761438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762505+0000","last_change":"2024-04-25T11:38:24.666747+0000","last_active":"2024-04-25T11:38:26.762505+0000","last_peered":"2024-04-25T11:38:26.762505+0000","last_clean":"2024-04-25T11:38:26.762505+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:38:26.762505+0000","last_undegraded":"2024-04-25T11:38:26.762505+0000","last_fullsized":"2024-04-25T11:38:26.762505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762221+0000","last_change":"2024-04-25T11:38:25.684128+0000","last_active":"2024-04-25T11:38:26.762221+0000","last_peered":"2024-04-25T11:38:26.762221+0000","last_clean":"2024-04-25T11:38:26.762221+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:38:26.762221+0000","last_undegraded":"2024-04-25T11:38:26.762221+0000","last_fullsized":"2024-04-25T11:38:26.762221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674047+0000","last_change":"2024-04-25T11:38:24.661103+0000","last_active":"2024-04-25T11:38:26.674047+0000","last_peered":"2024-04-25T11:38:26.674047+0000","last_clean":"2024-04-25T11:38:26.674047+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:38:26.674047+0000","last_undegraded":"2024-04-25T11:38:26.674047+0000","last_fullsized":"2024-04-25T11:38:26.674047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674380+0000","last_change":"2024-04-25T11:38:25.680490+0000","last_active":"2024-04-25T11:38:26.674380+0000","last_peered":"2024-04-25T11:38:26.674380+0000","last_clean":"2024-04-25T11:38:26.674380+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:38:26.674380+0000","last_undegraded":"2024-04-25T11:38:26.674380+0000","last_fullsized":"2024-04-25T11:38:26.674380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.678529+0000","last_change":"2024-04-25T11:38:24.665238+0000","last_active":"2024-04-25T11:38:30.678529+0000","last_peered":"2024-04-25T11:38:30.678529+0000","last_clean":"2024-04-25T11:38:30.678529+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:38:30.678529+0000","last_undegraded":"2024-04-25T11:38:30.678529+0000","last_fullsized":"2024-04-25T11:38:30.678529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761773+0000","last_change":"2024-04-25T11:38:25.682669+0000","last_active":"2024-04-25T11:38:26.761773+0000","last_peered":"2024-04-25T11:38:26.761773+0000","last_clean":"2024-04-25T11:38:26.761773+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:38:26.761773+0000","last_undegraded":"2024-04-25T11:38:26.761773+0000","last_fullsized":"2024-04-25T11:38:26.761773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.590779+0000","last_change":"2024-04-25T11:38:24.658666+0000","last_active":"2024-04-25T11:38:28.590779+0000","last_peered":"2024-04-25T11:38:28.590779+0000","last_clean":"2024-04-25T11:38:28.590779+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:38:28.590779+0000","last_undegraded":"2024-04-25T11:38:28.590779+0000","last_fullsized":"2024-04-25T11:38:28.590779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:24.295273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762494+0000","last_change":"2024-04-25T11:38:25.680783+0000","last_active":"2024-04-25T11:38:26.762494+0000","last_peered":"2024-04-25T11:38:26.762494+0000","last_clean":"2024-04-25T11:38:26.762494+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:38:26.762494+0000","last_undegraded":"2024-04-25T11:38:26.762494+0000","last_fullsized":"2024-04-25T11:38:26.762494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.675643+0000","last_change":"2024-04-25T11:38:24.656937+0000","last_active":"2024-04-25T11:38:30.675643+0000","last_peered":"2024-04-25T11:38:30.675643+0000","last_clean":"2024-04-25T11:38:30.675643+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:38:30.675643+0000","last_undegraded":"2024-04-25T11:38:30.675643+0000","last_fullsized":"2024-04-25T11:38:30.675643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763039+0000","last_change":"2024-04-25T11:38:25.681307+0000","last_active":"2024-04-25T11:38:26.763039+0000","last_peered":"2024-04-25T11:38:26.763039+0000","last_clean":"2024-04-25T11:38:26.763039+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:38:26.763039+0000","last_undegraded":"2024-04-25T11:38:26.763039+0000","last_fullsized":"2024-04-25T11:38:26.763039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:17.678183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.589301+0000","last_change":"2024-04-25T11:38:24.663247+0000","last_active":"2024-04-25T11:38:28.589301+0000","last_peered":"2024-04-25T11:38:28.589301+0000","last_clean":"2024-04-25T11:38:28.589301+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:38:28.589301+0000","last_undegraded":"2024-04-25T11:38:28.589301+0000","last_fullsized":"2024-04-25T11:38:28.589301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763176+0000","last_change":"2024-04-25T11:38:25.676409+0000","last_active":"2024-04-25T11:38:26.763176+0000","last_peered":"2024-04-25T11:38:26.763176+0000","last_clean":"2024-04-25T11:38:26.763176+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.763176+0000","last_undegraded":"2024-04-25T11:38:26.763176+0000","last_fullsized":"2024-04-25T11:38:26.763176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673831+0000","last_change":"2024-04-25T11:38:24.666800+0000","last_active":"2024-04-25T11:38:26.673831+0000","last_peered":"2024-04-25T11:38:26.673831+0000","last_clean":"2024-04-25T11:38:26.673831+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:38:26.673831+0000","last_undegraded":"2024-04-25T11:38:26.673831+0000","last_fullsized":"2024-04-25T11:38:26.673831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762286+0000","last_change":"2024-04-25T11:38:25.678136+0000","last_active":"2024-04-25T11:38:26.762286+0000","last_peered":"2024-04-25T11:38:26.762286+0000","last_clean":"2024-04-25T11:38:26.762286+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:38:26.762286+0000","last_undegraded":"2024-04-25T11:38:26.762286+0000","last_fullsized":"2024-04-25T11:38:26.762286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761547+0000","last_change":"2024-04-25T11:38:20.865911+0000","last_active":"2024-04-25T11:38:26.761547+0000","last_peered":"2024-04-25T11:38:26.761547+0000","last_clean":"2024-04-25T11:38:26.761547+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:38:26.761547+0000","last_undegraded":"2024-04-25T11:38:26.761547+0000","last_fullsized":"2024-04-25T11:38:26.761547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_clean_scrub_stamp":"2024-04-25T11:38:19.606342+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:58.126842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762328+0000","last_change":"2024-04-25T11:38:24.671283+0000","last_active":"2024-04-25T11:38:26.762328+0000","last_peered":"2024-04-25T11:38:26.762328+0000","last_clean":"2024-04-25T11:38:26.762328+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:38:26.762328+0000","last_undegraded":"2024-04-25T11:38:26.762328+0000","last_fullsized":"2024-04-25T11:38:26.762328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761567+0000","last_change":"2024-04-25T11:38:25.675002+0000","last_active":"2024-04-25T11:38:26.761567+0000","last_peered":"2024-04-25T11:38:26.761567+0000","last_clean":"2024-04-25T11:38:26.761567+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:38:26.761567+0000","last_undegraded":"2024-04-25T11:38:26.761567+0000","last_fullsized":"2024-04-25T11:38:26.761567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761489+0000","last_change":"2024-04-25T11:38:24.671213+0000","last_active":"2024-04-25T11:38:26.761489+0000","last_peered":"2024-04-25T11:38:26.761489+0000","last_clean":"2024-04-25T11:38:26.761489+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:38:26.761489+0000","last_undegraded":"2024-04-25T11:38:26.761489+0000","last_fullsized":"2024-04-25T11:38:26.761489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761052+0000","last_change":"2024-04-25T11:38:25.673353+0000","last_active":"2024-04-25T11:38:26.761052+0000","last_peered":"2024-04-25T11:38:26.761052+0000","last_clean":"2024-04-25T11:38:26.761052+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:38:26.761052+0000","last_undegraded":"2024-04-25T11:38:26.761052+0000","last_fullsized":"2024-04-25T11:38:26.761052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:48.553601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673682+0000","last_change":"2024-04-25T11:38:24.650599+0000","last_active":"2024-04-25T11:38:26.673682+0000","last_peered":"2024-04-25T11:38:26.673682+0000","last_clean":"2024-04-25T11:38:26.673682+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:38:26.673682+0000","last_undegraded":"2024-04-25T11:38:26.673682+0000","last_fullsized":"2024-04-25T11:38:26.673682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672982+0000","last_change":"2024-04-25T11:38:25.668956+0000","last_active":"2024-04-25T11:38:26.672982+0000","last_peered":"2024-04-25T11:38:26.672982+0000","last_clean":"2024-04-25T11:38:26.672982+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:38:26.672982+0000","last_undegraded":"2024-04-25T11:38:26.672982+0000","last_fullsized":"2024-04-25T11:38:26.672982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762612+0000","last_change":"2024-04-25T11:38:24.665496+0000","last_active":"2024-04-25T11:38:26.762612+0000","last_peered":"2024-04-25T11:38:26.762612+0000","last_clean":"2024-04-25T11:38:26.762612+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:38:26.762612+0000","last_undegraded":"2024-04-25T11:38:26.762612+0000","last_fullsized":"2024-04-25T11:38:26.762612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762812+0000","last_change":"2024-04-25T11:38:25.681800+0000","last_active":"2024-04-25T11:38:26.762812+0000","last_peered":"2024-04-25T11:38:26.762812+0000","last_clean":"2024-04-25T11:38:26.762812+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:38:26.762812+0000","last_undegraded":"2024-04-25T11:38:26.762812+0000","last_fullsized":"2024-04-25T11:38:26.762812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:39.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673142+0000","last_change":"2024-04-25T11:38:24.659792+0000","last_active":"2024-04-25T11:38:26.673142+0000","last_peered":"2024-04-25T11:38:26.673142+0000","last_clean":"2024-04-25T11:38:26.673142+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:38:26.673142+0000","last_undegraded":"2024-04-25T11:38:26.673142+0000","last_fullsized":"2024-04-25T11:38:26.673142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763580+0000","last_change":"2024-04-25T11:38:25.673411+0000","last_active":"2024-04-25T11:38:26.763580+0000","last_peered":"2024-04-25T11:38:26.763580+0000","last_clean":"2024-04-25T11:38:26.763580+0000","last_became_active":"2024-04-25T11:38:25.673116+0000","last_became_peered":"2024-04-25T11:38:25.673116+0000","last_unstale":"2024-04-25T11:38:26.763580+0000","last_undegraded":"2024-04-25T11:38:26.763580+0000","last_fullsized":"2024-04-25T11:38:26.763580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762671+0000","last_change":"2024-04-25T11:38:24.671412+0000","last_active":"2024-04-25T11:38:26.762671+0000","last_peered":"2024-04-25T11:38:26.762671+0000","last_clean":"2024-04-25T11:38:26.762671+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:38:26.762671+0000","last_undegraded":"2024-04-25T11:38:26.762671+0000","last_fullsized":"2024-04-25T11:38:26.762671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:34.607841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673048+0000","last_change":"2024-04-25T11:38:25.668959+0000","last_active":"2024-04-25T11:38:26.673048+0000","last_peered":"2024-04-25T11:38:26.673048+0000","last_clean":"2024-04-25T11:38:26.673048+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:38:26.673048+0000","last_undegraded":"2024-04-25T11:38:26.673048+0000","last_fullsized":"2024-04-25T11:38:26.673048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673459+0000","last_change":"2024-04-25T11:38:24.664146+0000","last_active":"2024-04-25T11:38:26.673459+0000","last_peered":"2024-04-25T11:38:26.673459+0000","last_clean":"2024-04-25T11:38:26.673459+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:38:26.673459+0000","last_undegraded":"2024-04-25T11:38:26.673459+0000","last_fullsized":"2024-04-25T11:38:26.673459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761339+0000","last_change":"2024-04-25T11:38:25.680105+0000","last_active":"2024-04-25T11:38:26.761339+0000","last_peered":"2024-04-25T11:38:26.761339+0000","last_clean":"2024-04-25T11:38:26.761339+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:38:26.761339+0000","last_undegraded":"2024-04-25T11:38:26.761339+0000","last_fullsized":"2024-04-25T11:38:26.761339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679496+0000","last_change":"2024-04-25T11:38:24.668369+0000","last_active":"2024-04-25T11:38:30.679496+0000","last_peered":"2024-04-25T11:38:30.679496+0000","last_clean":"2024-04-25T11:38:30.679496+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:38:30.679496+0000","last_undegraded":"2024-04-25T11:38:30.679496+0000","last_fullsized":"2024-04-25T11:38:30.679496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761723+0000","last_change":"2024-04-25T11:38:25.676446+0000","last_active":"2024-04-25T11:38:26.761723+0000","last_peered":"2024-04-25T11:38:26.761723+0000","last_clean":"2024-04-25T11:38:26.761723+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.761723+0000","last_undegraded":"2024-04-25T11:38:26.761723+0000","last_fullsized":"2024-04-25T11:38:26.761723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761379+0000","last_change":"2024-04-25T11:38:24.671951+0000","last_active":"2024-04-25T11:38:26.761379+0000","last_peered":"2024-04-25T11:38:26.761379+0000","last_clean":"2024-04-25T11:38:26.761379+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:38:26.761379+0000","last_undegraded":"2024-04-25T11:38:26.761379+0000","last_fullsized":"2024-04-25T11:38:26.761379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761852+0000","last_change":"2024-04-25T11:38:25.669474+0000","last_active":"2024-04-25T11:38:26.761852+0000","last_peered":"2024-04-25T11:38:26.761852+0000","last_clean":"2024-04-25T11:38:26.761852+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:38:26.761852+0000","last_undegraded":"2024-04-25T11:38:26.761852+0000","last_fullsized":"2024-04-25T11:38:26.761852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:38.359208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673212+0000","last_change":"2024-04-25T11:38:24.668689+0000","last_active":"2024-04-25T11:38:26.673212+0000","last_peered":"2024-04-25T11:38:26.673212+0000","last_clean":"2024-04-25T11:38:26.673212+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:38:26.673212+0000","last_undegraded":"2024-04-25T11:38:26.673212+0000","last_fullsized":"2024-04-25T11:38:26.673212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761421+0000","last_change":"2024-04-25T11:38:25.668088+0000","last_active":"2024-04-25T11:38:26.761421+0000","last_peered":"2024-04-25T11:38:26.761421+0000","last_clean":"2024-04-25T11:38:26.761421+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:38:26.761421+0000","last_undegraded":"2024-04-25T11:38:26.761421+0000","last_fullsized":"2024-04-25T11:38:26.761421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673373+0000","last_change":"2024-04-25T11:38:24.662913+0000","last_active":"2024-04-25T11:38:26.673373+0000","last_peered":"2024-04-25T11:38:26.673373+0000","last_clean":"2024-04-25T11:38:26.673373+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:38:26.673373+0000","last_undegraded":"2024-04-25T11:38:26.673373+0000","last_fullsized":"2024-04-25T11:38:26.673373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761308+0000","last_change":"2024-04-25T11:38:25.680244+0000","last_active":"2024-04-25T11:38:26.761308+0000","last_peered":"2024-04-25T11:38:26.761308+0000","last_clean":"2024-04-25T11:38:26.761308+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:38:26.761308+0000","last_undegraded":"2024-04-25T11:38:26.761308+0000","last_fullsized":"2024-04-25T11:38:26.761308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679126+0000","last_change":"2024-04-25T11:38:24.669196+0000","last_active":"2024-04-25T11:38:30.679126+0000","last_peered":"2024-04-25T11:38:30.679126+0000","last_clean":"2024-04-25T11:38:30.679126+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:38:30.679126+0000","last_undegraded":"2024-04-25T11:38:30.679126+0000","last_fullsized":"2024-04-25T11:38:30.679126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.791358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674198+0000","last_change":"2024-04-25T11:38:25.668400+0000","last_active":"2024-04-25T11:38:26.674198+0000","last_peered":"2024-04-25T11:38:26.674198+0000","last_clean":"2024-04-25T11:38:26.674198+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:38:26.674198+0000","last_undegraded":"2024-04-25T11:38:26.674198+0000","last_fullsized":"2024-04-25T11:38:26.674198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763520+0000","last_change":"2024-04-25T11:38:25.683663+0000","last_active":"2024-04-25T11:38:26.763520+0000","last_peered":"2024-04-25T11:38:26.763520+0000","last_clean":"2024-04-25T11:38:26.763520+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:38:26.763520+0000","last_undegraded":"2024-04-25T11:38:26.763520+0000","last_fullsized":"2024-04-25T11:38:26.763520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.680085+0000","last_change":"2024-04-25T11:38:24.670671+0000","last_active":"2024-04-25T11:38:30.680085+0000","last_peered":"2024-04-25T11:38:30.680085+0000","last_clean":"2024-04-25T11:38:30.680085+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:38:30.680085+0000","last_undegraded":"2024-04-25T11:38:30.680085+0000","last_fullsized":"2024-04-25T11:38:30.680085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763017+0000","last_change":"2024-04-25T11:38:25.681439+0000","last_active":"2024-04-25T11:38:26.763017+0000","last_peered":"2024-04-25T11:38:26.763017+0000","last_clean":"2024-04-25T11:38:26.763017+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:38:26.763017+0000","last_undegraded":"2024-04-25T11:38:26.763017+0000","last_fullsized":"2024-04-25T11:38:26.763017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679067+0000","last_change":"2024-04-25T11:38:24.669485+0000","last_active":"2024-04-25T11:38:30.679067+0000","last_peered":"2024-04-25T11:38:30.679067+0000","last_clean":"2024-04-25T11:38:30.679067+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:38:30.679067+0000","last_undegraded":"2024-04-25T11:38:30.679067+0000","last_fullsized":"2024-04-25T11:38:30.679067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761766+0000","last_change":"2024-04-25T11:38:25.684415+0000","last_active":"2024-04-25T11:38:26.761766+0000","last_peered":"2024-04-25T11:38:26.761766+0000","last_clean":"2024-04-25T11:38:26.761766+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:38:26.761766+0000","last_undegraded":"2024-04-25T11:38:26.761766+0000","last_fullsized":"2024-04-25T11:38:26.761766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.591003+0000","last_change":"2024-04-25T11:38:24.658741+0000","last_active":"2024-04-25T11:38:28.591003+0000","last_peered":"2024-04-25T11:38:28.591003+0000","last_clean":"2024-04-25T11:38:28.591003+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:38:28.591003+0000","last_undegraded":"2024-04-25T11:38:28.591003+0000","last_fullsized":"2024-04-25T11:38:28.591003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761168+0000","last_change":"2024-04-25T11:38:25.667505+0000","last_active":"2024-04-25T11:38:26.761168+0000","last_peered":"2024-04-25T11:38:26.761168+0000","last_clean":"2024-04-25T11:38:26.761168+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:38:26.761168+0000","last_undegraded":"2024-04-25T11:38:26.761168+0000","last_fullsized":"2024-04-25T11:38:26.761168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.680510+0000","last_change":"2024-04-25T11:38:24.669215+0000","last_active":"2024-04-25T11:38:30.680510+0000","last_peered":"2024-04-25T11:38:30.680510+0000","last_clean":"2024-04-25T11:38:30.680510+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:38:30.680510+0000","last_undegraded":"2024-04-25T11:38:30.680510+0000","last_fullsized":"2024-04-25T11:38:30.680510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":661884,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":661940,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:38:35.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:38:35.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.290+0000 7f1ffcc6b640 1 Processor -- start 2024-04-25T11:38:35.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.290+0000 7f1ffcc6b640 1 -- start start 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ffcc6b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ff8158470 0x7f1ff80ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ffcc6b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff80c8ce0 0x7f1ff815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ffcc6b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff8070730 con 0x7f1ff80b0ec0 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ffcc6b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff815b1a0 con 0x7f1ff8158470 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ffcc6b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff80ca370 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ff659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ff8158470 0x7f1ff80ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ff6d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1ff80b0ec0 0x7f1ff80ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44436/0 (socket says 172.21.15.40:44436) 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ff6d9d640 1 -- 172.21.15.40:0/2291467478 learned_addr learned my addr 172.21.15.40:0/2291467478 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ff5d9b640 1 --2- 172.21.15.40:0/2291467478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff80c8ce0 0x7f1ff815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ff659c640 1 -- 172.21.15.40:0/2291467478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff80c8ce0 msgr2=0x7f1ff815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ff659c640 1 --2- 172.21.15.40:0/2291467478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff80c8ce0 0x7f1ff815ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ff659c640 1 -- 172.21.15.40:0/2291467478 >> v1:172.21.15.40:6789/0 conn(0x7f1ff80b0ec0 legacy=0x7f1ff80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ff559a640 1 -- 172.21.15.40:0/2291467478 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464679786 0 0) 0x7f1ff8070730 con 0x7f1ff80b0ec0 2024-04-25T11:38:35.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.291+0000 7f1ff659c640 1 -- 172.21.15.40:0/2291467478 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff8070450 con 0x7f1ff8158470 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.292+0000 7f1ff659c640 1 --2- 172.21.15.40:0/2291467478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff8158470 0x7f1ff80ca800 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00097a0 tx=0x7f1ff002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d764061bf20d8585 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.292+0000 7f1ff559a640 1 -- 172.21.15.40:0/2291467478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0037070 con 0x7f1ff8158470 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.292+0000 7f1ff559a640 1 -- 172.21.15.40:0/2291467478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff0032440 con 0x7f1ff8158470 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.292+0000 7f1ff559a640 1 -- 172.21.15.40:0/2291467478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0032810 con 0x7f1ff8158470 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.293+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/2291467478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff8158470 msgr2=0x7f1ff80ca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.293+0000 7f1ffcc6b640 1 --2- 172.21.15.40:0/2291467478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff8158470 0x7f1ff80ca800 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00097a0 tx=0x7f1ff002a210 comp rx=0 tx=0).stop 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.293+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/2291467478 shutdown_connections 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.293+0000 7f1ffcc6b640 1 --2- 172.21.15.40:0/2291467478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff80c8ce0 0x7f1ff815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.293+0000 7f1ffcc6b640 1 --2- 172.21.15.40:0/2291467478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff8158470 0x7f1ff80ca800 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.293+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/2291467478 >> 172.21.15.40:0/2291467478 conn(0x7f1ff8067ad0 msgr2=0x7f1ff8067ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.293+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/2291467478 shutdown_connections 2024-04-25T11:38:35.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.293+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/2291467478 wait complete. 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.294+0000 7f1ffcc6b640 1 Processor -- start 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.294+0000 7f1ffcc6b640 1 -- start start 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ffcc6b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff80b0ec0 0x7f1ff80cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ffcc6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff80c8ce0 0x7f1ff80afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ffcc6b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff8158470 0x7f1ff80b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ffcc6b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff806d0e0 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ffcc6b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff806ce00 con 0x7f1ff80b0ec0 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ffcc6b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff8070080 con 0x7f1ff8158470 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff80c8ce0 0x7f1ff80afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff80c8ce0 0x7f1ff80afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53544/0 (socket says 172.21.15.40:53544) 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff5d9b640 1 -- 172.21.15.40:0/3703105693 learned_addr learned my addr 172.21.15.40:0/3703105693 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff659c640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff80b0ec0 0x7f1ff80cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff6d9d640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff8158470 0x7f1ff80b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff5d9b640 1 -- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff8158470 msgr2=0x7f1ff80b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff5d9b640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff8158470 0x7f1ff80b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff5d9b640 1 -- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff80b0ec0 msgr2=0x7f1ff80cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff5d9b640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff80b0ec0 0x7f1ff80cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.295+0000 7f1ff5d9b640 1 -- 172.21.15.40:0/3703105693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff81f0930 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.296+0000 7f1ff5d9b640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff80c8ce0 0x7f1ff80afc60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000d830 tx=0x7f1fe000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.296+0000 7f1feeffd640 1 -- 172.21.15.40:0/3703105693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000b6c0 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.296+0000 7f1feeffd640 1 -- 172.21.15.40:0/3703105693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe0019070 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.296+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff81f0c10 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.296+0000 7f1feeffd640 1 -- 172.21.15.40:0/3703105693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000ba80 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.296+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff81f11d0 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.297+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff8068bb0 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.297+0000 7f1feeffd640 1 -- 172.21.15.40:0/3703105693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fe000bc20 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.298+0000 7f1feeffd640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1fd806c8f0 0x7f1fd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.298+0000 7f1feeffd640 1 -- 172.21.15.40:0/3703105693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1fe008be10 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.298+0000 7f1ff659c640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1fd806c8f0 0x7f1fd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.298+0000 7f1ff659c640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1fd806c8f0 0x7f1fd806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0034000 tx=0x7f1ff002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.302+0000 7f1feeffd640 1 -- 172.21.15.40:0/3703105693 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fe0059c80 con 0x7f1ff80c8ce0 2024-04-25T11:38:35.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.485+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ff8156280 con 0x7f1fd806c8f0 2024-04-25T11:38:35.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.492+0000 7f1feeffd640 1 -- 172.21.15.40:0/3703105693 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251912 (secure 0 0 0) 0x7f1ff8156280 con 0x7f1fd806c8f0 2024-04-25T11:38:35.481 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:35.483 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:38:35.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1fd806c8f0 msgr2=0x7f1fd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1fd806c8f0 0x7f1fd806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0034000 tx=0x7f1ff002e000 comp rx=0 tx=0).stop 2024-04-25T11:38:35.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff80c8ce0 msgr2=0x7f1ff80afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff80c8ce0 0x7f1ff80afc60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000d830 tx=0x7f1fe000dcf0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 shutdown_connections 2024-04-25T11:38:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1fd806c8f0 0x7f1fd806edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff8158470 0x7f1ff80b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ff80c8ce0 0x7f1ff80afc60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 --2- 172.21.15.40:0/3703105693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff80b0ec0 0x7f1ff80cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 >> 172.21.15.40:0/3703105693 conn(0x7f1ff8067ad0 msgr2=0x7f1ff81565f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 shutdown_connections 2024-04-25T11:38:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.495+0000 7f1ffcc6b640 1 -- 172.21.15.40:0/3703105693 wait complete. 2024-04-25T11:38:35.493 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T11:38:34.953029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217920,"kb_used_data":1792,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756800,"statfs":{"total":773094113280,"available":772870963200,"internally_reserved":0,"allocated":1835008,"data_stored":1753028,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.297728"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762879+0000","last_change":"2024-04-25T11:38:25.677572+0000","last_active":"2024-04-25T11:38:26.762879+0000","last_peered":"2024-04-25T11:38:26.762879+0000","last_clean":"2024-04-25T11:38:26.762879+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:38:26.762879+0000","last_undegraded":"2024-04-25T11:38:26.762879+0000","last_fullsized":"2024-04-25T11:38:26.762879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673367+0000","last_change":"2024-04-25T11:38:25.677437+0000","last_active":"2024-04-25T11:38:26.673367+0000","last_peered":"2024-04-25T11:38:26.673367+0000","last_clean":"2024-04-25T11:38:26.673367+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:38:26.673367+0000","last_undegraded":"2024-04-25T11:38:26.673367+0000","last_fullsized":"2024-04-25T11:38:26.673367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761847+0000","last_change":"2024-04-25T11:38:25.676553+0000","last_active":"2024-04-25T11:38:26.761847+0000","last_peered":"2024-04-25T11:38:26.761847+0000","last_clean":"2024-04-25T11:38:26.761847+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:38:26.761847+0000","last_undegraded":"2024-04-25T11:38:26.761847+0000","last_fullsized":"2024-04-25T11:38:26.761847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762862+0000","last_change":"2024-04-25T11:38:25.676518+0000","last_active":"2024-04-25T11:38:26.762862+0000","last_peered":"2024-04-25T11:38:26.762862+0000","last_clean":"2024-04-25T11:38:26.762862+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:38:26.762862+0000","last_undegraded":"2024-04-25T11:38:26.762862+0000","last_fullsized":"2024-04-25T11:38:26.762862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762062+0000","last_change":"2024-04-25T11:38:25.678524+0000","last_active":"2024-04-25T11:38:26.762062+0000","last_peered":"2024-04-25T11:38:26.762062+0000","last_clean":"2024-04-25T11:38:26.762062+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:38:26.762062+0000","last_undegraded":"2024-04-25T11:38:26.762062+0000","last_fullsized":"2024-04-25T11:38:26.762062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761774+0000","last_change":"2024-04-25T11:38:25.683983+0000","last_active":"2024-04-25T11:38:26.761774+0000","last_peered":"2024-04-25T11:38:26.761774+0000","last_clean":"2024-04-25T11:38:26.761774+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:38:26.761774+0000","last_undegraded":"2024-04-25T11:38:26.761774+0000","last_fullsized":"2024-04-25T11:38:26.761774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761720+0000","last_change":"2024-04-25T11:38:25.682904+0000","last_active":"2024-04-25T11:38:26.761720+0000","last_peered":"2024-04-25T11:38:26.761720+0000","last_clean":"2024-04-25T11:38:26.761720+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:38:26.761720+0000","last_undegraded":"2024-04-25T11:38:26.761720+0000","last_fullsized":"2024-04-25T11:38:26.761720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761274+0000","last_change":"2024-04-25T11:38:25.670312+0000","last_active":"2024-04-25T11:38:26.761274+0000","last_peered":"2024-04-25T11:38:26.761274+0000","last_clean":"2024-04-25T11:38:26.761274+0000","last_became_active":"2024-04-25T11:38:25.669988+0000","last_became_peered":"2024-04-25T11:38:25.669988+0000","last_unstale":"2024-04-25T11:38:26.761274+0000","last_undegraded":"2024-04-25T11:38:26.761274+0000","last_fullsized":"2024-04-25T11:38:26.761274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762939+0000","last_change":"2024-04-25T11:38:25.678459+0000","last_active":"2024-04-25T11:38:26.762939+0000","last_peered":"2024-04-25T11:38:26.762939+0000","last_clean":"2024-04-25T11:38:26.762939+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:38:26.762939+0000","last_undegraded":"2024-04-25T11:38:26.762939+0000","last_fullsized":"2024-04-25T11:38:26.762939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674629+0000","last_change":"2024-04-25T11:38:25.675561+0000","last_active":"2024-04-25T11:38:26.674629+0000","last_peered":"2024-04-25T11:38:26.674629+0000","last_clean":"2024-04-25T11:38:26.674629+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:38:26.674629+0000","last_undegraded":"2024-04-25T11:38:26.674629+0000","last_fullsized":"2024-04-25T11:38:26.674629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674251+0000","last_change":"2024-04-25T11:38:25.671113+0000","last_active":"2024-04-25T11:38:26.674251+0000","last_peered":"2024-04-25T11:38:26.674251+0000","last_clean":"2024-04-25T11:38:26.674251+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:38:26.674251+0000","last_undegraded":"2024-04-25T11:38:26.674251+0000","last_fullsized":"2024-04-25T11:38:26.674251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762821+0000","last_change":"2024-04-25T11:38:25.665690+0000","last_active":"2024-04-25T11:38:26.762821+0000","last_peered":"2024-04-25T11:38:26.762821+0000","last_clean":"2024-04-25T11:38:26.762821+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:38:26.762821+0000","last_undegraded":"2024-04-25T11:38:26.762821+0000","last_fullsized":"2024-04-25T11:38:26.762821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:25.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761152+0000","last_change":"2024-04-25T11:38:25.666858+0000","last_active":"2024-04-25T11:38:26.761152+0000","last_peered":"2024-04-25T11:38:26.761152+0000","last_clean":"2024-04-25T11:38:26.761152+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:38:26.761152+0000","last_undegraded":"2024-04-25T11:38:26.761152+0000","last_fullsized":"2024-04-25T11:38:26.761152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761293+0000","last_change":"2024-04-25T11:38:25.677561+0000","last_active":"2024-04-25T11:38:26.761293+0000","last_peered":"2024-04-25T11:38:26.761293+0000","last_clean":"2024-04-25T11:38:26.761293+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:38:26.761293+0000","last_undegraded":"2024-04-25T11:38:26.761293+0000","last_fullsized":"2024-04-25T11:38:26.761293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673120+0000","last_change":"2024-04-25T11:38:25.672129+0000","last_active":"2024-04-25T11:38:26.673120+0000","last_peered":"2024-04-25T11:38:26.673120+0000","last_clean":"2024-04-25T11:38:26.673120+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:38:26.673120+0000","last_undegraded":"2024-04-25T11:38:26.673120+0000","last_fullsized":"2024-04-25T11:38:26.673120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761886+0000","last_change":"2024-04-25T11:38:25.682547+0000","last_active":"2024-04-25T11:38:26.761886+0000","last_peered":"2024-04-25T11:38:26.761886+0000","last_clean":"2024-04-25T11:38:26.761886+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:38:26.761886+0000","last_undegraded":"2024-04-25T11:38:26.761886+0000","last_fullsized":"2024-04-25T11:38:26.761886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763750+0000","last_change":"2024-04-25T11:38:25.683798+0000","last_active":"2024-04-25T11:38:26.763750+0000","last_peered":"2024-04-25T11:38:26.763750+0000","last_clean":"2024-04-25T11:38:26.763750+0000","last_became_active":"2024-04-25T11:38:25.682590+0000","last_became_peered":"2024-04-25T11:38:25.682590+0000","last_unstale":"2024-04-25T11:38:26.763750+0000","last_undegraded":"2024-04-25T11:38:26.763750+0000","last_fullsized":"2024-04-25T11:38:26.763750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673869+0000","last_change":"2024-04-25T11:38:25.669650+0000","last_active":"2024-04-25T11:38:26.673869+0000","last_peered":"2024-04-25T11:38:26.673869+0000","last_clean":"2024-04-25T11:38:26.673869+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:38:26.673869+0000","last_undegraded":"2024-04-25T11:38:26.673869+0000","last_fullsized":"2024-04-25T11:38:26.673869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762614+0000","last_change":"2024-04-25T11:38:25.678238+0000","last_active":"2024-04-25T11:38:26.762614+0000","last_peered":"2024-04-25T11:38:26.762614+0000","last_clean":"2024-04-25T11:38:26.762614+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:38:26.762614+0000","last_undegraded":"2024-04-25T11:38:26.762614+0000","last_fullsized":"2024-04-25T11:38:26.762614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674167+0000","last_change":"2024-04-25T11:38:25.666736+0000","last_active":"2024-04-25T11:38:26.674167+0000","last_peered":"2024-04-25T11:38:26.674167+0000","last_clean":"2024-04-25T11:38:26.674167+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:38:26.674167+0000","last_undegraded":"2024-04-25T11:38:26.674167+0000","last_fullsized":"2024-04-25T11:38:26.674167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761661+0000","last_change":"2024-04-25T11:38:25.680587+0000","last_active":"2024-04-25T11:38:26.761661+0000","last_peered":"2024-04-25T11:38:26.761661+0000","last_clean":"2024-04-25T11:38:26.761661+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:38:26.761661+0000","last_undegraded":"2024-04-25T11:38:26.761661+0000","last_fullsized":"2024-04-25T11:38:26.761661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762157+0000","last_change":"2024-04-25T11:38:25.680464+0000","last_active":"2024-04-25T11:38:26.762157+0000","last_peered":"2024-04-25T11:38:26.762157+0000","last_clean":"2024-04-25T11:38:26.762157+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:38:26.762157+0000","last_undegraded":"2024-04-25T11:38:26.762157+0000","last_fullsized":"2024-04-25T11:38:26.762157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761201+0000","last_change":"2024-04-25T11:38:25.672189+0000","last_active":"2024-04-25T11:38:26.761201+0000","last_peered":"2024-04-25T11:38:26.761201+0000","last_clean":"2024-04-25T11:38:26.761201+0000","last_became_active":"2024-04-25T11:38:25.671920+0000","last_became_peered":"2024-04-25T11:38:25.671920+0000","last_unstale":"2024-04-25T11:38:26.761201+0000","last_undegraded":"2024-04-25T11:38:26.761201+0000","last_fullsized":"2024-04-25T11:38:26.761201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762072+0000","last_change":"2024-04-25T11:38:25.674108+0000","last_active":"2024-04-25T11:38:26.762072+0000","last_peered":"2024-04-25T11:38:26.762072+0000","last_clean":"2024-04-25T11:38:26.762072+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:38:26.762072+0000","last_undegraded":"2024-04-25T11:38:26.762072+0000","last_fullsized":"2024-04-25T11:38:26.762072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761259+0000","last_change":"2024-04-25T11:38:25.672215+0000","last_active":"2024-04-25T11:38:26.761259+0000","last_peered":"2024-04-25T11:38:26.761259+0000","last_clean":"2024-04-25T11:38:26.761259+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:38:26.761259+0000","last_undegraded":"2024-04-25T11:38:26.761259+0000","last_fullsized":"2024-04-25T11:38:26.761259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674666+0000","last_change":"2024-04-25T11:38:25.679772+0000","last_active":"2024-04-25T11:38:26.674666+0000","last_peered":"2024-04-25T11:38:26.674666+0000","last_clean":"2024-04-25T11:38:26.674666+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:38:26.674666+0000","last_undegraded":"2024-04-25T11:38:26.674666+0000","last_fullsized":"2024-04-25T11:38:26.674666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673164+0000","last_change":"2024-04-25T11:38:25.665626+0000","last_active":"2024-04-25T11:38:26.673164+0000","last_peered":"2024-04-25T11:38:26.673164+0000","last_clean":"2024-04-25T11:38:26.673164+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:38:26.673164+0000","last_undegraded":"2024-04-25T11:38:26.673164+0000","last_fullsized":"2024-04-25T11:38:26.673164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761385+0000","last_change":"2024-04-25T11:38:25.672935+0000","last_active":"2024-04-25T11:38:26.761385+0000","last_peered":"2024-04-25T11:38:26.761385+0000","last_clean":"2024-04-25T11:38:26.761385+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:38:26.761385+0000","last_undegraded":"2024-04-25T11:38:26.761385+0000","last_fullsized":"2024-04-25T11:38:26.761385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761842+0000","last_change":"2024-04-25T11:38:25.679486+0000","last_active":"2024-04-25T11:38:26.761842+0000","last_peered":"2024-04-25T11:38:26.761842+0000","last_clean":"2024-04-25T11:38:26.761842+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:38:26.761842+0000","last_undegraded":"2024-04-25T11:38:26.761842+0000","last_fullsized":"2024-04-25T11:38:26.761842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762406+0000","last_change":"2024-04-25T11:38:25.679437+0000","last_active":"2024-04-25T11:38:26.762406+0000","last_peered":"2024-04-25T11:38:26.762406+0000","last_clean":"2024-04-25T11:38:26.762406+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:38:26.762406+0000","last_undegraded":"2024-04-25T11:38:26.762406+0000","last_fullsized":"2024-04-25T11:38:26.762406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673702+0000","last_change":"2024-04-25T11:38:25.670384+0000","last_active":"2024-04-25T11:38:26.673702+0000","last_peered":"2024-04-25T11:38:26.673702+0000","last_clean":"2024-04-25T11:38:26.673702+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:38:26.673702+0000","last_undegraded":"2024-04-25T11:38:26.673702+0000","last_fullsized":"2024-04-25T11:38:26.673702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674250+0000","last_change":"2024-04-25T11:38:25.680252+0000","last_active":"2024-04-25T11:38:26.674250+0000","last_peered":"2024-04-25T11:38:26.674250+0000","last_clean":"2024-04-25T11:38:26.674250+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:38:26.674250+0000","last_undegraded":"2024-04-25T11:38:26.674250+0000","last_fullsized":"2024-04-25T11:38:26.674250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762291+0000","last_change":"2024-04-25T11:38:25.669542+0000","last_active":"2024-04-25T11:38:26.762291+0000","last_peered":"2024-04-25T11:38:26.762291+0000","last_clean":"2024-04-25T11:38:26.762291+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:38:26.762291+0000","last_undegraded":"2024-04-25T11:38:26.762291+0000","last_fullsized":"2024-04-25T11:38:26.762291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:51.929330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.590981+0000","last_change":"2024-04-25T11:38:24.666197+0000","last_active":"2024-04-25T11:38:28.590981+0000","last_peered":"2024-04-25T11:38:28.590981+0000","last_clean":"2024-04-25T11:38:28.590981+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:38:28.590981+0000","last_undegraded":"2024-04-25T11:38:28.590981+0000","last_fullsized":"2024-04-25T11:38:28.590981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.198972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761974+0000","last_change":"2024-04-25T11:38:25.682931+0000","last_active":"2024-04-25T11:38:26.761974+0000","last_peered":"2024-04-25T11:38:26.761974+0000","last_clean":"2024-04-25T11:38:26.761974+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:38:26.761974+0000","last_undegraded":"2024-04-25T11:38:26.761974+0000","last_fullsized":"2024-04-25T11:38:26.761974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.590777+0000","last_change":"2024-04-25T11:38:24.660888+0000","last_active":"2024-04-25T11:38:28.590777+0000","last_peered":"2024-04-25T11:38:28.590777+0000","last_clean":"2024-04-25T11:38:28.590777+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:38:28.590777+0000","last_undegraded":"2024-04-25T11:38:28.590777+0000","last_fullsized":"2024-04-25T11:38:28.590777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761513+0000","last_change":"2024-04-25T11:38:24.657815+0000","last_active":"2024-04-25T11:38:26.761513+0000","last_peered":"2024-04-25T11:38:26.761513+0000","last_clean":"2024-04-25T11:38:26.761513+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:38:26.761513+0000","last_undegraded":"2024-04-25T11:38:26.761513+0000","last_fullsized":"2024-04-25T11:38:26.761513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761437+0000","last_change":"2024-04-25T11:38:25.680969+0000","last_active":"2024-04-25T11:38:26.761437+0000","last_peered":"2024-04-25T11:38:26.761437+0000","last_clean":"2024-04-25T11:38:26.761437+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:38:26.761437+0000","last_undegraded":"2024-04-25T11:38:26.761437+0000","last_fullsized":"2024-04-25T11:38:26.761437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762182+0000","last_change":"2024-04-25T11:38:24.668899+0000","last_active":"2024-04-25T11:38:26.762182+0000","last_peered":"2024-04-25T11:38:26.762182+0000","last_clean":"2024-04-25T11:38:26.762182+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:38:26.762182+0000","last_undegraded":"2024-04-25T11:38:26.762182+0000","last_fullsized":"2024-04-25T11:38:26.762182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672975+0000","last_change":"2024-04-25T11:38:25.666511+0000","last_active":"2024-04-25T11:38:26.672975+0000","last_peered":"2024-04-25T11:38:26.672975+0000","last_clean":"2024-04-25T11:38:26.672975+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:38:26.672975+0000","last_undegraded":"2024-04-25T11:38:26.672975+0000","last_fullsized":"2024-04-25T11:38:26.672975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674478+0000","last_change":"2024-04-25T11:38:24.666735+0000","last_active":"2024-04-25T11:38:26.674478+0000","last_peered":"2024-04-25T11:38:26.674478+0000","last_clean":"2024-04-25T11:38:26.674478+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:38:26.674478+0000","last_undegraded":"2024-04-25T11:38:26.674478+0000","last_fullsized":"2024-04-25T11:38:26.674478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763586+0000","last_change":"2024-04-25T11:38:25.680065+0000","last_active":"2024-04-25T11:38:26.763586+0000","last_peered":"2024-04-25T11:38:26.763586+0000","last_clean":"2024-04-25T11:38:26.763586+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:38:26.763586+0000","last_undegraded":"2024-04-25T11:38:26.763586+0000","last_fullsized":"2024-04-25T11:38:26.763586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763667+0000","last_change":"2024-04-25T11:38:24.671566+0000","last_active":"2024-04-25T11:38:26.763667+0000","last_peered":"2024-04-25T11:38:26.763667+0000","last_clean":"2024-04-25T11:38:26.763667+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:38:26.763667+0000","last_undegraded":"2024-04-25T11:38:26.763667+0000","last_fullsized":"2024-04-25T11:38:26.763667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761910+0000","last_change":"2024-04-25T11:38:25.678075+0000","last_active":"2024-04-25T11:38:26.761910+0000","last_peered":"2024-04-25T11:38:26.761910+0000","last_clean":"2024-04-25T11:38:26.761910+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:38:26.761910+0000","last_undegraded":"2024-04-25T11:38:26.761910+0000","last_fullsized":"2024-04-25T11:38:26.761910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.678792+0000","last_change":"2024-04-25T11:38:24.667282+0000","last_active":"2024-04-25T11:38:30.678792+0000","last_peered":"2024-04-25T11:38:30.678792+0000","last_clean":"2024-04-25T11:38:30.678792+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:38:30.678792+0000","last_undegraded":"2024-04-25T11:38:30.678792+0000","last_fullsized":"2024-04-25T11:38:30.678792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762833+0000","last_change":"2024-04-25T11:38:25.680417+0000","last_active":"2024-04-25T11:38:26.762833+0000","last_peered":"2024-04-25T11:38:26.762833+0000","last_clean":"2024-04-25T11:38:26.762833+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:38:26.762833+0000","last_undegraded":"2024-04-25T11:38:26.762833+0000","last_fullsized":"2024-04-25T11:38:26.762833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.680566+0000","last_change":"2024-04-25T11:38:24.662562+0000","last_active":"2024-04-25T11:38:30.680566+0000","last_peered":"2024-04-25T11:38:30.680566+0000","last_clean":"2024-04-25T11:38:30.680566+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:38:30.680566+0000","last_undegraded":"2024-04-25T11:38:30.680566+0000","last_fullsized":"2024-04-25T11:38:30.680566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762135+0000","last_change":"2024-04-25T11:38:25.677518+0000","last_active":"2024-04-25T11:38:26.762135+0000","last_peered":"2024-04-25T11:38:26.762135+0000","last_clean":"2024-04-25T11:38:26.762135+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:38:26.762135+0000","last_undegraded":"2024-04-25T11:38:26.762135+0000","last_fullsized":"2024-04-25T11:38:26.762135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679081+0000","last_change":"2024-04-25T11:38:24.658960+0000","last_active":"2024-04-25T11:38:30.679081+0000","last_peered":"2024-04-25T11:38:30.679081+0000","last_clean":"2024-04-25T11:38:30.679081+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:38:30.679081+0000","last_undegraded":"2024-04-25T11:38:30.679081+0000","last_fullsized":"2024-04-25T11:38:30.679081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761438+0000","last_change":"2024-04-25T11:38:25.681355+0000","last_active":"2024-04-25T11:38:26.761438+0000","last_peered":"2024-04-25T11:38:26.761438+0000","last_clean":"2024-04-25T11:38:26.761438+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:38:26.761438+0000","last_undegraded":"2024-04-25T11:38:26.761438+0000","last_fullsized":"2024-04-25T11:38:26.761438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762505+0000","last_change":"2024-04-25T11:38:24.666747+0000","last_active":"2024-04-25T11:38:26.762505+0000","last_peered":"2024-04-25T11:38:26.762505+0000","last_clean":"2024-04-25T11:38:26.762505+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:38:26.762505+0000","last_undegraded":"2024-04-25T11:38:26.762505+0000","last_fullsized":"2024-04-25T11:38:26.762505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762221+0000","last_change":"2024-04-25T11:38:25.684128+0000","last_active":"2024-04-25T11:38:26.762221+0000","last_peered":"2024-04-25T11:38:26.762221+0000","last_clean":"2024-04-25T11:38:26.762221+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:38:26.762221+0000","last_undegraded":"2024-04-25T11:38:26.762221+0000","last_fullsized":"2024-04-25T11:38:26.762221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674047+0000","last_change":"2024-04-25T11:38:24.661103+0000","last_active":"2024-04-25T11:38:26.674047+0000","last_peered":"2024-04-25T11:38:26.674047+0000","last_clean":"2024-04-25T11:38:26.674047+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:38:26.674047+0000","last_undegraded":"2024-04-25T11:38:26.674047+0000","last_fullsized":"2024-04-25T11:38:26.674047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674380+0000","last_change":"2024-04-25T11:38:25.680490+0000","last_active":"2024-04-25T11:38:26.674380+0000","last_peered":"2024-04-25T11:38:26.674380+0000","last_clean":"2024-04-25T11:38:26.674380+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:38:26.674380+0000","last_undegraded":"2024-04-25T11:38:26.674380+0000","last_fullsized":"2024-04-25T11:38:26.674380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.678529+0000","last_change":"2024-04-25T11:38:24.665238+0000","last_active":"2024-04-25T11:38:30.678529+0000","last_peered":"2024-04-25T11:38:30.678529+0000","last_clean":"2024-04-25T11:38:30.678529+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:38:30.678529+0000","last_undegraded":"2024-04-25T11:38:30.678529+0000","last_fullsized":"2024-04-25T11:38:30.678529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761773+0000","last_change":"2024-04-25T11:38:25.682669+0000","last_active":"2024-04-25T11:38:26.761773+0000","last_peered":"2024-04-25T11:38:26.761773+0000","last_clean":"2024-04-25T11:38:26.761773+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:38:26.761773+0000","last_undegraded":"2024-04-25T11:38:26.761773+0000","last_fullsized":"2024-04-25T11:38:26.761773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.590779+0000","last_change":"2024-04-25T11:38:24.658666+0000","last_active":"2024-04-25T11:38:28.590779+0000","last_peered":"2024-04-25T11:38:28.590779+0000","last_clean":"2024-04-25T11:38:28.590779+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:38:28.590779+0000","last_undegraded":"2024-04-25T11:38:28.590779+0000","last_fullsized":"2024-04-25T11:38:28.590779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:24.295273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762494+0000","last_change":"2024-04-25T11:38:25.680783+0000","last_active":"2024-04-25T11:38:26.762494+0000","last_peered":"2024-04-25T11:38:26.762494+0000","last_clean":"2024-04-25T11:38:26.762494+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:38:26.762494+0000","last_undegraded":"2024-04-25T11:38:26.762494+0000","last_fullsized":"2024-04-25T11:38:26.762494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.675643+0000","last_change":"2024-04-25T11:38:24.656937+0000","last_active":"2024-04-25T11:38:30.675643+0000","last_peered":"2024-04-25T11:38:30.675643+0000","last_clean":"2024-04-25T11:38:30.675643+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:38:30.675643+0000","last_undegraded":"2024-04-25T11:38:30.675643+0000","last_fullsized":"2024-04-25T11:38:30.675643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763039+0000","last_change":"2024-04-25T11:38:25.681307+0000","last_active":"2024-04-25T11:38:26.763039+0000","last_peered":"2024-04-25T11:38:26.763039+0000","last_clean":"2024-04-25T11:38:26.763039+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:38:26.763039+0000","last_undegraded":"2024-04-25T11:38:26.763039+0000","last_fullsized":"2024-04-25T11:38:26.763039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:17.678183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.589301+0000","last_change":"2024-04-25T11:38:24.663247+0000","last_active":"2024-04-25T11:38:28.589301+0000","last_peered":"2024-04-25T11:38:28.589301+0000","last_clean":"2024-04-25T11:38:28.589301+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:38:28.589301+0000","last_undegraded":"2024-04-25T11:38:28.589301+0000","last_fullsized":"2024-04-25T11:38:28.589301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763176+0000","last_change":"2024-04-25T11:38:25.676409+0000","last_active":"2024-04-25T11:38:26.763176+0000","last_peered":"2024-04-25T11:38:26.763176+0000","last_clean":"2024-04-25T11:38:26.763176+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.763176+0000","last_undegraded":"2024-04-25T11:38:26.763176+0000","last_fullsized":"2024-04-25T11:38:26.763176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673831+0000","last_change":"2024-04-25T11:38:24.666800+0000","last_active":"2024-04-25T11:38:26.673831+0000","last_peered":"2024-04-25T11:38:26.673831+0000","last_clean":"2024-04-25T11:38:26.673831+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:38:26.673831+0000","last_undegraded":"2024-04-25T11:38:26.673831+0000","last_fullsized":"2024-04-25T11:38:26.673831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762286+0000","last_change":"2024-04-25T11:38:25.678136+0000","last_active":"2024-04-25T11:38:26.762286+0000","last_peered":"2024-04-25T11:38:26.762286+0000","last_clean":"2024-04-25T11:38:26.762286+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:38:26.762286+0000","last_undegraded":"2024-04-25T11:38:26.762286+0000","last_fullsized":"2024-04-25T11:38:26.762286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761547+0000","last_change":"2024-04-25T11:38:20.865911+0000","last_active":"2024-04-25T11:38:26.761547+0000","last_peered":"2024-04-25T11:38:26.761547+0000","last_clean":"2024-04-25T11:38:26.761547+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:38:26.761547+0000","last_undegraded":"2024-04-25T11:38:26.761547+0000","last_fullsized":"2024-04-25T11:38:26.761547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_clean_scrub_stamp":"2024-04-25T11:38:19.606342+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:58.126842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762328+0000","last_change":"2024-04-25T11:38:24.671283+0000","last_active":"2024-04-25T11:38:26.762328+0000","last_peered":"2024-04-25T11:38:26.762328+0000","last_clean":"2024-04-25T11:38:26.762328+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:38:26.762328+0000","last_undegraded":"2024-04-25T11:38:26.762328+0000","last_fullsized":"2024-04-25T11:38:26.762328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761567+0000","last_change":"2024-04-25T11:38:25.675002+0000","last_active":"2024-04-25T11:38:26.761567+0000","last_peered":"2024-04-25T11:38:26.761567+0000","last_clean":"2024-04-25T11:38:26.761567+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:38:26.761567+0000","last_undegraded":"2024-04-25T11:38:26.761567+0000","last_fullsized":"2024-04-25T11:38:26.761567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761489+0000","last_change":"2024-04-25T11:38:24.671213+0000","last_active":"2024-04-25T11:38:26.761489+0000","last_peered":"2024-04-25T11:38:26.761489+0000","last_clean":"2024-04-25T11:38:26.761489+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:38:26.761489+0000","last_undegraded":"2024-04-25T11:38:26.761489+0000","last_fullsized":"2024-04-25T11:38:26.761489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761052+0000","last_change":"2024-04-25T11:38:25.673353+0000","last_active":"2024-04-25T11:38:26.761052+0000","last_peered":"2024-04-25T11:38:26.761052+0000","last_clean":"2024-04-25T11:38:26.761052+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:38:26.761052+0000","last_undegraded":"2024-04-25T11:38:26.761052+0000","last_fullsized":"2024-04-25T11:38:26.761052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:48.553601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673682+0000","last_change":"2024-04-25T11:38:24.650599+0000","last_active":"2024-04-25T11:38:26.673682+0000","last_peered":"2024-04-25T11:38:26.673682+0000","last_clean":"2024-04-25T11:38:26.673682+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:38:26.673682+0000","last_undegraded":"2024-04-25T11:38:26.673682+0000","last_fullsized":"2024-04-25T11:38:26.673682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672982+0000","last_change":"2024-04-25T11:38:25.668956+0000","last_active":"2024-04-25T11:38:26.672982+0000","last_peered":"2024-04-25T11:38:26.672982+0000","last_clean":"2024-04-25T11:38:26.672982+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:38:26.672982+0000","last_undegraded":"2024-04-25T11:38:26.672982+0000","last_fullsized":"2024-04-25T11:38:26.672982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762612+0000","last_change":"2024-04-25T11:38:24.665496+0000","last_active":"2024-04-25T11:38:26.762612+0000","last_peered":"2024-04-25T11:38:26.762612+0000","last_clean":"2024-04-25T11:38:26.762612+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:38:26.762612+0000","last_undegraded":"2024-04-25T11:38:26.762612+0000","last_fullsized":"2024-04-25T11:38:26.762612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762812+0000","last_change":"2024-04-25T11:38:25.681800+0000","last_active":"2024-04-25T11:38:26.762812+0000","last_peered":"2024-04-25T11:38:26.762812+0000","last_clean":"2024-04-25T11:38:26.762812+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:38:26.762812+0000","last_undegraded":"2024-04-25T11:38:26.762812+0000","last_fullsized":"2024-04-25T11:38:26.762812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:39.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673142+0000","last_change":"2024-04-25T11:38:24.659792+0000","last_active":"2024-04-25T11:38:26.673142+0000","last_peered":"2024-04-25T11:38:26.673142+0000","last_clean":"2024-04-25T11:38:26.673142+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:38:26.673142+0000","last_undegraded":"2024-04-25T11:38:26.673142+0000","last_fullsized":"2024-04-25T11:38:26.673142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763580+0000","last_change":"2024-04-25T11:38:25.673411+0000","last_active":"2024-04-25T11:38:26.763580+0000","last_peered":"2024-04-25T11:38:26.763580+0000","last_clean":"2024-04-25T11:38:26.763580+0000","last_became_active":"2024-04-25T11:38:25.673116+0000","last_became_peered":"2024-04-25T11:38:25.673116+0000","last_unstale":"2024-04-25T11:38:26.763580+0000","last_undegraded":"2024-04-25T11:38:26.763580+0000","last_fullsized":"2024-04-25T11:38:26.763580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762671+0000","last_change":"2024-04-25T11:38:24.671412+0000","last_active":"2024-04-25T11:38:26.762671+0000","last_peered":"2024-04-25T11:38:26.762671+0000","last_clean":"2024-04-25T11:38:26.762671+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:38:26.762671+0000","last_undegraded":"2024-04-25T11:38:26.762671+0000","last_fullsized":"2024-04-25T11:38:26.762671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:34.607841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673048+0000","last_change":"2024-04-25T11:38:25.668959+0000","last_active":"2024-04-25T11:38:26.673048+0000","last_peered":"2024-04-25T11:38:26.673048+0000","last_clean":"2024-04-25T11:38:26.673048+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:38:26.673048+0000","last_undegraded":"2024-04-25T11:38:26.673048+0000","last_fullsized":"2024-04-25T11:38:26.673048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673459+0000","last_change":"2024-04-25T11:38:24.664146+0000","last_active":"2024-04-25T11:38:26.673459+0000","last_peered":"2024-04-25T11:38:26.673459+0000","last_clean":"2024-04-25T11:38:26.673459+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:38:26.673459+0000","last_undegraded":"2024-04-25T11:38:26.673459+0000","last_fullsized":"2024-04-25T11:38:26.673459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761339+0000","last_change":"2024-04-25T11:38:25.680105+0000","last_active":"2024-04-25T11:38:26.761339+0000","last_peered":"2024-04-25T11:38:26.761339+0000","last_clean":"2024-04-25T11:38:26.761339+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:38:26.761339+0000","last_undegraded":"2024-04-25T11:38:26.761339+0000","last_fullsized":"2024-04-25T11:38:26.761339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679496+0000","last_change":"2024-04-25T11:38:24.668369+0000","last_active":"2024-04-25T11:38:30.679496+0000","last_peered":"2024-04-25T11:38:30.679496+0000","last_clean":"2024-04-25T11:38:30.679496+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:38:30.679496+0000","last_undegraded":"2024-04-25T11:38:30.679496+0000","last_fullsized":"2024-04-25T11:38:30.679496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761723+0000","last_change":"2024-04-25T11:38:25.676446+0000","last_active":"2024-04-25T11:38:26.761723+0000","last_peered":"2024-04-25T11:38:26.761723+0000","last_clean":"2024-04-25T11:38:26.761723+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.761723+0000","last_undegraded":"2024-04-25T11:38:26.761723+0000","last_fullsized":"2024-04-25T11:38:26.761723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761379+0000","last_change":"2024-04-25T11:38:24.671951+0000","last_active":"2024-04-25T11:38:26.761379+0000","last_peered":"2024-04-25T11:38:26.761379+0000","last_clean":"2024-04-25T11:38:26.761379+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:38:26.761379+0000","last_undegraded":"2024-04-25T11:38:26.761379+0000","last_fullsized":"2024-04-25T11:38:26.761379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761852+0000","last_change":"2024-04-25T11:38:25.669474+0000","last_active":"2024-04-25T11:38:26.761852+0000","last_peered":"2024-04-25T11:38:26.761852+0000","last_clean":"2024-04-25T11:38:26.761852+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:38:26.761852+0000","last_undegraded":"2024-04-25T11:38:26.761852+0000","last_fullsized":"2024-04-25T11:38:26.761852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:38.359208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673212+0000","last_change":"2024-04-25T11:38:24.668689+0000","last_active":"2024-04-25T11:38:26.673212+0000","last_peered":"2024-04-25T11:38:26.673212+0000","last_clean":"2024-04-25T11:38:26.673212+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:38:26.673212+0000","last_undegraded":"2024-04-25T11:38:26.673212+0000","last_fullsized":"2024-04-25T11:38:26.673212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761421+0000","last_change":"2024-04-25T11:38:25.668088+0000","last_active":"2024-04-25T11:38:26.761421+0000","last_peered":"2024-04-25T11:38:26.761421+0000","last_clean":"2024-04-25T11:38:26.761421+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:38:26.761421+0000","last_undegraded":"2024-04-25T11:38:26.761421+0000","last_fullsized":"2024-04-25T11:38:26.761421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673373+0000","last_change":"2024-04-25T11:38:24.662913+0000","last_active":"2024-04-25T11:38:26.673373+0000","last_peered":"2024-04-25T11:38:26.673373+0000","last_clean":"2024-04-25T11:38:26.673373+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:38:26.673373+0000","last_undegraded":"2024-04-25T11:38:26.673373+0000","last_fullsized":"2024-04-25T11:38:26.673373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761308+0000","last_change":"2024-04-25T11:38:25.680244+0000","last_active":"2024-04-25T11:38:26.761308+0000","last_peered":"2024-04-25T11:38:26.761308+0000","last_clean":"2024-04-25T11:38:26.761308+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:38:26.761308+0000","last_undegraded":"2024-04-25T11:38:26.761308+0000","last_fullsized":"2024-04-25T11:38:26.761308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679126+0000","last_change":"2024-04-25T11:38:24.669196+0000","last_active":"2024-04-25T11:38:30.679126+0000","last_peered":"2024-04-25T11:38:30.679126+0000","last_clean":"2024-04-25T11:38:30.679126+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:38:30.679126+0000","last_undegraded":"2024-04-25T11:38:30.679126+0000","last_fullsized":"2024-04-25T11:38:30.679126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.791358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674198+0000","last_change":"2024-04-25T11:38:25.668400+0000","last_active":"2024-04-25T11:38:26.674198+0000","last_peered":"2024-04-25T11:38:26.674198+0000","last_clean":"2024-04-25T11:38:26.674198+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:38:26.674198+0000","last_undegraded":"2024-04-25T11:38:26.674198+0000","last_fullsized":"2024-04-25T11:38:26.674198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763520+0000","last_change":"2024-04-25T11:38:25.683663+0000","last_active":"2024-04-25T11:38:26.763520+0000","last_peered":"2024-04-25T11:38:26.763520+0000","last_clean":"2024-04-25T11:38:26.763520+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:38:26.763520+0000","last_undegraded":"2024-04-25T11:38:26.763520+0000","last_fullsized":"2024-04-25T11:38:26.763520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.680085+0000","last_change":"2024-04-25T11:38:24.670671+0000","last_active":"2024-04-25T11:38:30.680085+0000","last_peered":"2024-04-25T11:38:30.680085+0000","last_clean":"2024-04-25T11:38:30.680085+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:38:30.680085+0000","last_undegraded":"2024-04-25T11:38:30.680085+0000","last_fullsized":"2024-04-25T11:38:30.680085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763017+0000","last_change":"2024-04-25T11:38:25.681439+0000","last_active":"2024-04-25T11:38:26.763017+0000","last_peered":"2024-04-25T11:38:26.763017+0000","last_clean":"2024-04-25T11:38:26.763017+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:38:26.763017+0000","last_undegraded":"2024-04-25T11:38:26.763017+0000","last_fullsized":"2024-04-25T11:38:26.763017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679067+0000","last_change":"2024-04-25T11:38:24.669485+0000","last_active":"2024-04-25T11:38:30.679067+0000","last_peered":"2024-04-25T11:38:30.679067+0000","last_clean":"2024-04-25T11:38:30.679067+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:38:30.679067+0000","last_undegraded":"2024-04-25T11:38:30.679067+0000","last_fullsized":"2024-04-25T11:38:30.679067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761766+0000","last_change":"2024-04-25T11:38:25.684415+0000","last_active":"2024-04-25T11:38:26.761766+0000","last_peered":"2024-04-25T11:38:26.761766+0000","last_clean":"2024-04-25T11:38:26.761766+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:38:26.761766+0000","last_undegraded":"2024-04-25T11:38:26.761766+0000","last_fullsized":"2024-04-25T11:38:26.761766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.591003+0000","last_change":"2024-04-25T11:38:24.658741+0000","last_active":"2024-04-25T11:38:28.591003+0000","last_peered":"2024-04-25T11:38:28.591003+0000","last_clean":"2024-04-25T11:38:28.591003+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:38:28.591003+0000","last_undegraded":"2024-04-25T11:38:28.591003+0000","last_fullsized":"2024-04-25T11:38:28.591003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761168+0000","last_change":"2024-04-25T11:38:25.667505+0000","last_active":"2024-04-25T11:38:26.761168+0000","last_peered":"2024-04-25T11:38:26.761168+0000","last_clean":"2024-04-25T11:38:26.761168+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:38:26.761168+0000","last_undegraded":"2024-04-25T11:38:26.761168+0000","last_fullsized":"2024-04-25T11:38:26.761168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.680510+0000","last_change":"2024-04-25T11:38:24.669215+0000","last_active":"2024-04-25T11:38:30.680510+0000","last_peered":"2024-04-25T11:38:30.680510+0000","last_clean":"2024-04-25T11:38:30.680510+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:38:30.680510+0000","last_undegraded":"2024-04-25T11:38:30.680510+0000","last_fullsized":"2024-04-25T11:38:30.680510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":661884,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":661940,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:38:35.497 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:38:35.497 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T11:38:35.497 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T11:38:35.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:38:35.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.645+0000 7f0112ba2640 1 Processor -- start 2024-04-25T11:38:35.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.646+0000 7f0112ba2640 1 -- start start 2024-04-25T11:38:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.646+0000 7f0112ba2640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f010c0cd410 0x7f010c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.646+0000 7f0112ba2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f010c15ae60 con 0x7f010c0cb240 2024-04-25T11:38:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.646+0000 7f0112ba2640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f010c15b140 con 0x7f010c0c7b70 2024-04-25T11:38:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.646+0000 7f0112ba2640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f010c15b420 con 0x7f010c0cd410 2024-04-25T11:38:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.646+0000 7f011093e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f010c0cb240 0x7f010c0cc010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44448/0 (socket says 172.21.15.40:44448) 2024-04-25T11:38:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.646+0000 7f011093e640 1 -- 172.21.15.40:0/1976505937 learned_addr learned my addr 172.21.15.40:0/1976505937 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.647+0000 7f011113f640 1 --2- 172.21.15.40:0/1976505937 >> v2:172.21.15.62:3300/0 conn(0x7f010c0cd410 0x7f010c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.647+0000 7f01037fe640 1 -- 172.21.15.40:0/1976505937 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657322336 0 0) 0x7f010c15ae60 con 0x7f010c0cb240 2024-04-25T11:38:35.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.647+0000 7f01037fe640 1 -- 172.21.15.40:0/1976505937 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00f4003520 con 0x7f010c0cb240 2024-04-25T11:38:35.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.647+0000 7f011113f640 1 -- 172.21.15.40:0/1976505937 >> v1:172.21.15.62:6789/0 conn(0x7f010c0c7b70 legacy=0x7f010c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.647+0000 7f01037fe640 1 -- 172.21.15.40:0/1976505937 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 319181668 0 0) 0x7f00f4003520 con 0x7f010c0cb240 2024-04-25T11:38:35.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.647+0000 7f011113f640 1 -- 172.21.15.40:0/1976505937 >> v1:172.21.15.40:6789/0 conn(0x7f010c0cb240 legacy=0x7f010c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.647+0000 7f011113f640 1 -- 172.21.15.40:0/1976505937 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c15b700 con 0x7f010c0cd410 2024-04-25T11:38:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f011113f640 1 --2- 172.21.15.40:0/1976505937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c0cd410 0x7f010c0cb620 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0104009d90 tx=0x7f010402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b219ceb4b03ee18d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f01037fe640 1 -- 172.21.15.40:0/1976505937 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108001670 con 0x7f010c0cd410 2024-04-25T11:38:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f01037fe640 1 -- 172.21.15.40:0/1976505937 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0104007540 con 0x7f010c0cd410 2024-04-25T11:38:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f01037fe640 1 -- 172.21.15.40:0/1976505937 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104007910 con 0x7f010c0cd410 2024-04-25T11:38:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f0112ba2640 1 -- 172.21.15.40:0/1976505937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c0cd410 msgr2=0x7f010c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f0112ba2640 1 --2- 172.21.15.40:0/1976505937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c0cd410 0x7f010c0cb620 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0104009d90 tx=0x7f010402a680 comp rx=0 tx=0).stop 2024-04-25T11:38:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f0112ba2640 1 -- 172.21.15.40:0/1976505937 shutdown_connections 2024-04-25T11:38:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f0112ba2640 1 --2- 172.21.15.40:0/1976505937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c0cd410 0x7f010c0cb620 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f0112ba2640 1 -- 172.21.15.40:0/1976505937 >> 172.21.15.40:0/1976505937 conn(0x7f010c06c3d0 msgr2=0x7f010c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.648+0000 7f0112ba2640 1 -- 172.21.15.40:0/1976505937 shutdown_connections 2024-04-25T11:38:35.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.649+0000 7f0112ba2640 1 -- 172.21.15.40:0/1976505937 wait complete. 2024-04-25T11:38:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.650+0000 7f0112ba2640 1 Processor -- start 2024-04-25T11:38:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.650+0000 7f0112ba2640 1 -- start start 2024-04-25T11:38:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.650+0000 7f0112ba2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c0c7b70 0x7f010c151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.650+0000 7f0112ba2640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f010c0cb240 0x7f010c1f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.650+0000 7f0112ba2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f010c0cd410 0x7f010c0c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.650+0000 7f0112ba2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1552b0 con 0x7f010c0cd410 2024-04-25T11:38:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.650+0000 7f0112ba2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f010c0b6290 con 0x7f010c0c7b70 2024-04-25T11:38:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f0112ba2640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f010c0c5440 con 0x7f010c0cb240 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011113f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f010c0cd410 0x7f010c0c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011113f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f010c0cd410 0x7f010c0c40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53552/0 (socket says 172.21.15.40:53552) 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011113f640 1 -- 172.21.15.40:0/2175833045 learned_addr learned my addr 172.21.15.40:0/2175833045 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011093e640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c0c7b70 0x7f010c151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f0103fff640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f010c0cb240 0x7f010c1f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011113f640 1 -- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f010c0cb240 msgr2=0x7f010c1f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011113f640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f010c0cb240 0x7f010c1f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011113f640 1 -- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c0c7b70 msgr2=0x7f010c151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011113f640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c0c7b70 0x7f010c151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011113f640 1 -- 172.21.15.40:0/2175833045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c1f0c90 con 0x7f010c0cd410 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f011113f640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f010c0cd410 0x7f010c0c40e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f010402ac70 tx=0x7f0104004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f01017fa640 1 -- 172.21.15.40:0/2175833045 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104040070 con 0x7f010c0cd410 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f01017fa640 1 -- 172.21.15.40:0/2175833045 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0104031c40 con 0x7f010c0cd410 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c1559c0 con 0x7f010c0cd410 2024-04-25T11:38:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f01017fa640 1 -- 172.21.15.40:0/2175833045 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104009450 con 0x7f010c0cd410 2024-04-25T11:38:35.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.651+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c155c80 con 0x7f010c0cd410 2024-04-25T11:38:35.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.652+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c068bb0 con 0x7f010c0cd410 2024-04-25T11:38:35.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.653+0000 7f01017fa640 1 -- 172.21.15.40:0/2175833045 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0104039040 con 0x7f010c0cd410 2024-04-25T11:38:35.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.653+0000 7f01017fa640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f00f806c8f0 0x7f00f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.653+0000 7f01017fa640 1 -- 172.21.15.40:0/2175833045 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f01040ace70 con 0x7f010c0cd410 2024-04-25T11:38:35.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.653+0000 7f011093e640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f00f806c8f0 0x7f00f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.654+0000 7f011093e640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f00f806c8f0 0x7f00f806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0108005880 tx=0x7f0108000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.657+0000 7f01017fa640 1 -- 172.21.15.40:0/2175833045 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f010407abc0 con 0x7f010c0cd410 2024-04-25T11:38:35.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.886+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f010c154a00 con 0x7f010c0cd410 2024-04-25T11:38:35.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.886+0000 7f01017fa640 1 -- 172.21.15.40:0/2175833045 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f010407eec0 con 0x7f010c0cd410 2024-04-25T11:38:35.873 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:35.874 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:38:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f00f806c8f0 msgr2=0x7f00f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f00f806c8f0 0x7f00f806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0108005880 tx=0x7f0108000bc0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f010c0cd410 msgr2=0x7f010c0c40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f010c0cd410 0x7f010c0c40e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f010402ac70 tx=0x7f0104004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 shutdown_connections 2024-04-25T11:38:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f00f806c8f0 0x7f00f806edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f010c0cd410 0x7f010c0c40e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f010c0cb240 0x7f010c1f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 --2- 172.21.15.40:0/2175833045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c0c7b70 0x7f010c151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 >> 172.21.15.40:0/2175833045 conn(0x7f010c06c3d0 msgr2=0x7f010c14f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 shutdown_connections 2024-04-25T11:38:35.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:35.888+0000 7f0112ba2640 1 -- 172.21.15.40:0/2175833045 wait complete. 2024-04-25T11:38:38.887 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:38:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.013+0000 7fe376394640 1 Processor -- start 2024-04-25T11:38:39.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.014+0000 7fe376394640 1 -- start start 2024-04-25T11:38:39.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.014+0000 7fe376394640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe370156130 0x7fe3700c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.014+0000 7fe376394640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3700c9640 con 0x7fe37006fee0 2024-04-25T11:38:39.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.014+0000 7fe376394640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3700ca2e0 con 0x7fe37015c0f0 2024-04-25T11:38:39.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.014+0000 7fe376394640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe3700ca5c0 con 0x7fe370156130 2024-04-25T11:38:39.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.015+0000 7fe36ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe370156130 0x7fe3700c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.015+0000 7fe36ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe370156130 0x7fe3700c51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53562/0 (socket says 172.21.15.40:53562) 2024-04-25T11:38:39.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.015+0000 7fe36ffff640 1 -- 172.21.15.40:0/21128665 learned_addr learned my addr 172.21.15.40:0/21128665 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:39.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.015+0000 7fe374931640 1 --1- 172.21.15.40:0/21128665 >> v1:172.21.15.40:6789/0 conn(0x7fe37006fee0 0x7fe370163420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44464/0 (socket says 172.21.15.40:44464) 2024-04-25T11:38:39.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.015+0000 7fe36ffff640 1 -- 172.21.15.40:0/21128665 >> v1:172.21.15.62:6789/0 conn(0x7fe37015c0f0 legacy=0x7fe3700c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.015+0000 7fe36effd640 1 -- 172.21.15.40:0/21128665 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1775520130 0 0) 0x7fe3700ca2e0 con 0x7fe37015c0f0 2024-04-25T11:38:39.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.015+0000 7fe36ffff640 1 -- 172.21.15.40:0/21128665 >> v1:172.21.15.40:6789/0 conn(0x7fe37006fee0 legacy=0x7fe370163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.015+0000 7fe36ffff640 1 -- 172.21.15.40:0/21128665 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3700ca8a0 con 0x7fe370156130 2024-04-25T11:38:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe36ffff640 1 --2- 172.21.15.40:0/21128665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe370156130 0x7fe3700c51c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe360009870 tx=0x7fe36002a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551246acf17452de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe36effd640 1 -- 172.21.15.40:0/21128665 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360037070 con 0x7fe370156130 2024-04-25T11:38:39.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe36effd640 1 -- 172.21.15.40:0/21128665 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe360032640 con 0x7fe370156130 2024-04-25T11:38:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe36effd640 1 -- 172.21.15.40:0/21128665 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360032a10 con 0x7fe370156130 2024-04-25T11:38:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe376394640 1 -- 172.21.15.40:0/21128665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe370156130 msgr2=0x7fe3700c51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe376394640 1 --2- 172.21.15.40:0/21128665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe370156130 0x7fe3700c51c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe360009870 tx=0x7fe36002a3c0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe376394640 1 -- 172.21.15.40:0/21128665 shutdown_connections 2024-04-25T11:38:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe376394640 1 --2- 172.21.15.40:0/21128665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe370156130 0x7fe3700c51c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe376394640 1 -- 172.21.15.40:0/21128665 >> 172.21.15.40:0/21128665 conn(0x7fe37006bd90 msgr2=0x7fe3700676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe376394640 1 -- 172.21.15.40:0/21128665 shutdown_connections 2024-04-25T11:38:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.016+0000 7fe376394640 1 -- 172.21.15.40:0/21128665 wait complete. 2024-04-25T11:38:39.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.017+0000 7fe376394640 1 Processor -- start 2024-04-25T11:38:39.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.018+0000 7fe376394640 1 -- start start 2024-04-25T11:38:39.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.018+0000 7fe376394640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe37006fee0 0x7fe37015af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.018+0000 7fe376394640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe370156130 0x7fe3700cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.018+0000 7fe376394640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37015c0f0 0x7fe3701f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.018+0000 7fe376394640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe3700cbb90 con 0x7fe37015c0f0 2024-04-25T11:38:39.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.018+0000 7fe376394640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3700ce640 con 0x7fe37006fee0 2024-04-25T11:38:39.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.018+0000 7fe376394640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe370156510 con 0x7fe370156130 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.018+0000 7fe374931640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37015c0f0 0x7fe3701f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe374931640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37015c0f0 0x7fe3701f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53570/0 (socket says 172.21.15.40:53570) 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe374931640 1 -- 172.21.15.40:0/1476077306 learned_addr learned my addr 172.21.15.40:0/1476077306 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe36ffff640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe37006fee0 0x7fe37015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe36f7fe640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe370156130 0x7fe3700cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe374931640 1 -- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe370156130 msgr2=0x7fe3700cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe374931640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe370156130 0x7fe3700cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe374931640 1 -- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe37006fee0 msgr2=0x7fe37015af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe374931640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe37006fee0 0x7fe37015af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe374931640 1 -- 172.21.15.40:0/1476077306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe370153b10 con 0x7fe37015c0f0 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe374931640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37015c0f0 0x7fe3701f1e80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe36800c7d0 tx=0x7fe36800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe36cff9640 1 -- 172.21.15.40:0/1476077306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368017070 con 0x7fe37015c0f0 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe36cff9640 1 -- 172.21.15.40:0/1476077306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36800a640 con 0x7fe37015c0f0 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3701f28a0 con 0x7fe37015c0f0 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe36cff9640 1 -- 172.21.15.40:0/1476077306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36800aa30 con 0x7fe37015c0f0 2024-04-25T11:38:39.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.019+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3701eeb20 con 0x7fe37015c0f0 2024-04-25T11:38:39.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.020+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe370155c90 con 0x7fe37015c0f0 2024-04-25T11:38:39.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.021+0000 7fe36cff9640 1 -- 172.21.15.40:0/1476077306 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe36801a070 con 0x7fe37015c0f0 2024-04-25T11:38:39.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.021+0000 7fe36cff9640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe35c06c920 0x7fe35c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.021+0000 7fe36cff9640 1 -- 172.21.15.40:0/1476077306 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe36808b170 con 0x7fe37015c0f0 2024-04-25T11:38:39.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.021+0000 7fe36ffff640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe35c06c920 0x7fe35c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.022+0000 7fe36ffff640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe35c06c920 0x7fe35c06ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe360034000 tx=0x7fe36002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:39.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.026+0000 7fe36cff9640 1 -- 172.21.15.40:0/1476077306 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe370155c90 con 0x7fe37015c0f0 2024-04-25T11:38:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.249+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe3701ee910 con 0x7fe37015c0f0 2024-04-25T11:38:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.250+0000 7fe36cff9640 1 -- 172.21.15.40:0/1476077306 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7fe368058ec0 con 0x7fe37015c0f0 2024-04-25T11:38:39.237 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:39.237 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe35c06c920 msgr2=0x7fe35c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe35c06c920 0x7fe35c06ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe360034000 tx=0x7fe36002e000 comp rx=0 tx=0).stop 2024-04-25T11:38:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37015c0f0 msgr2=0x7fe3701f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37015c0f0 0x7fe3701f1e80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe36800c7d0 tx=0x7fe36800cc90 comp rx=0 tx=0).stop 2024-04-25T11:38:39.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 shutdown_connections 2024-04-25T11:38:39.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe35c06c920 0x7fe35c06ede0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe37015c0f0 0x7fe3701f1e80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe370156130 0x7fe3700cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 --2- 172.21.15.40:0/1476077306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe37006fee0 0x7fe37015af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 >> 172.21.15.40:0/1476077306 conn(0x7fe37006bd90 msgr2=0x7fe370154660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:39.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 shutdown_connections 2024-04-25T11:38:39.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:39.252+0000 7fe376394640 1 -- 172.21.15.40:0/1476077306 wait complete. 2024-04-25T11:38:40.254 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-04-25T11:38:40.267+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:38:40.254 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-04-25T11:38:40.267+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T11:38:42.251 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:38:42.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.376+0000 7f912135d640 1 Processor -- start 2024-04-25T11:38:42.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.376+0000 7f912135d640 1 -- start start 2024-04-25T11:38:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.377+0000 7f912135d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c06fee0 0x7f911c070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.377+0000 7f912135d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f911c0c9640 con 0x7f911c0c51c0 2024-04-25T11:38:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.377+0000 7f912135d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f911c0ca2e0 con 0x7f911c156130 2024-04-25T11:38:42.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.377+0000 7f912135d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f911c0ca5c0 con 0x7f911c06fee0 2024-04-25T11:38:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.377+0000 7f911b7fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c06fee0 0x7f911c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.377+0000 7f911affd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f911c156130 0x7f911c15c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:38326/0 (socket says 172.21.15.40:38326) 2024-04-25T11:38:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.378+0000 7f911affd640 1 -- 172.21.15.40:0/2071819879 learned_addr learned my addr 172.21.15.40:0/2071819879 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.378+0000 7f9119ffb640 1 -- 172.21.15.40:0/2071819879 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184402170 0 0) 0x7f911c0ca2e0 con 0x7f911c156130 2024-04-25T11:38:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.378+0000 7f9119ffb640 1 -- 172.21.15.40:0/2071819879 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9108003520 con 0x7f911c156130 2024-04-25T11:38:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.378+0000 7f9119ffb640 1 -- 172.21.15.40:0/2071819879 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2026202276 0 0) 0x7f911c0c9640 con 0x7f911c0c51c0 2024-04-25T11:38:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.378+0000 7f911b7fe640 1 -- 172.21.15.40:0/2071819879 >> v1:172.21.15.62:6789/0 conn(0x7f911c156130 legacy=0x7f911c15c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.378+0000 7f911b7fe640 1 -- 172.21.15.40:0/2071819879 >> v1:172.21.15.40:6789/0 conn(0x7f911c0c51c0 legacy=0x7f911c0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.378+0000 7f911b7fe640 1 -- 172.21.15.40:0/2071819879 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c0ca8a0 con 0x7f911c06fee0 2024-04-25T11:38:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f911b7fe640 1 --2- 172.21.15.40:0/2071819879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c06fee0 0x7f911c070330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f91140097d0 tx=0x7f911402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9032128b4f5664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f9119ffb640 1 -- 172.21.15.40:0/2071819879 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114037070 con 0x7f911c06fee0 2024-04-25T11:38:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f9119ffb640 1 -- 172.21.15.40:0/2071819879 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9114032440 con 0x7f911c06fee0 2024-04-25T11:38:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f9119ffb640 1 -- 172.21.15.40:0/2071819879 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91140327f0 con 0x7f911c06fee0 2024-04-25T11:38:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f912135d640 1 -- 172.21.15.40:0/2071819879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c06fee0 msgr2=0x7f911c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f912135d640 1 --2- 172.21.15.40:0/2071819879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c06fee0 0x7f911c070330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f91140097d0 tx=0x7f911402a290 comp rx=0 tx=0).stop 2024-04-25T11:38:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f912135d640 1 -- 172.21.15.40:0/2071819879 shutdown_connections 2024-04-25T11:38:42.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f912135d640 1 --2- 172.21.15.40:0/2071819879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c06fee0 0x7f911c070330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f912135d640 1 -- 172.21.15.40:0/2071819879 >> 172.21.15.40:0/2071819879 conn(0x7f911c06bd90 msgr2=0x7f911c0676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:42.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f912135d640 1 -- 172.21.15.40:0/2071819879 shutdown_connections 2024-04-25T11:38:42.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.379+0000 7f912135d640 1 -- 172.21.15.40:0/2071819879 wait complete. 2024-04-25T11:38:42.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.381+0000 7f912135d640 1 Processor -- start 2024-04-25T11:38:42.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.381+0000 7f912135d640 1 -- start start 2024-04-25T11:38:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f912135d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f911c06fee0 0x7f911c1f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f912135d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c0c51c0 0x7f911c1e4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f912135d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c156130 0x7f911c1e4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f912135d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0cbb90 con 0x7f911c156130 2024-04-25T11:38:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f912135d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0ce640 con 0x7f911c06fee0 2024-04-25T11:38:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f912135d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f911c156510 con 0x7f911c0c51c0 2024-04-25T11:38:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c156130 0x7f911c1e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:42.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f911c06fee0 0x7f911c1f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911a7fc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c0c51c0 0x7f911c1e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c156130 0x7f911c1e4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53578/0 (socket says 172.21.15.40:53578) 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f911c06fee0 0x7f911c1f20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:52254/0 (socket says 172.21.15.40:52254) 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911b7fe640 1 -- 172.21.15.40:0/1863564682 learned_addr learned my addr 172.21.15.40:0/1863564682 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911b7fe640 1 -- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c0c51c0 msgr2=0x7f911c1e4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911b7fe640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c0c51c0 0x7f911c1e4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911b7fe640 1 -- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f911c06fee0 msgr2=0x7f911c1f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911b7fe640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f911c06fee0 0x7f911c1f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911b7fe640 1 -- 172.21.15.40:0/1863564682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c1553b0 con 0x7f911c156130 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911affd640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f911c06fee0 0x7f911c1f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911a7fc640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c0c51c0 0x7f911c1e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f911b7fe640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c156130 0x7f911c1e4950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9114009720 tx=0x7f91140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f91137fe640 1 -- 172.21.15.40:0/1863564682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114037040 con 0x7f911c156130 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f91137fe640 1 -- 172.21.15.40:0/1863564682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9114004230 con 0x7f911c156130 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c15aba0 con 0x7f911c156130 2024-04-25T11:38:42.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.382+0000 7f91137fe640 1 -- 172.21.15.40:0/1863564682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114003610 con 0x7f911c156130 2024-04-25T11:38:42.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.383+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c15a160 con 0x7f911c156130 2024-04-25T11:38:42.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.383+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911c069200 con 0x7f911c156130 2024-04-25T11:38:42.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.384+0000 7f91137fe640 1 -- 172.21.15.40:0/1863564682 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91140037b0 con 0x7f911c156130 2024-04-25T11:38:42.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.385+0000 7f91137fe640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f90fc06c920 0x7f90fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.385+0000 7f911affd640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f90fc06c920 0x7f90fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:42.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.385+0000 7f91137fe640 1 -- 172.21.15.40:0/1863564682 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f91140ac680 con 0x7f911c156130 2024-04-25T11:38:42.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.385+0000 7f911affd640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f90fc06c920 0x7f90fc06ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f910c005c00 tx=0x7f910c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:42.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.389+0000 7f91137fe640 1 -- 172.21.15.40:0/1863564682 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f911407a4f0 con 0x7f911c156130 2024-04-25T11:38:42.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.614+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f911c070a50 con 0x7f911c156130 2024-04-25T11:38:42.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.614+0000 7f91137fe640 1 -- 172.21.15.40:0/1863564682 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f911407e7f0 con 0x7f911c156130 2024-04-25T11:38:42.602 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:42.602 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f90fc06c920 msgr2=0x7f90fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f90fc06c920 0x7f90fc06ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f910c005c00 tx=0x7f910c009040 comp rx=0 tx=0).stop 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c156130 msgr2=0x7f911c1e4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c156130 0x7f911c1e4950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9114009720 tx=0x7f91140046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 shutdown_connections 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f90fc06c920 0x7f90fc06ede0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c156130 0x7f911c1e4950 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f911c0c51c0 0x7f911c1e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 --2- 172.21.15.40:0/1863564682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f911c06fee0 0x7f911c1f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 >> 172.21.15.40:0/1863564682 conn(0x7f911c06bd90 msgr2=0x7f911c0676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 shutdown_connections 2024-04-25T11:38:42.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:42.616+0000 7f912135d640 1 -- 172.21.15.40:0/1863564682 wait complete. 2024-04-25T11:38:45.254 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-04-25T11:38:45.267+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T11:38:45.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:38:45.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.740+0000 7fe431d20640 1 Processor -- start 2024-04-25T11:38:45.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.740+0000 7fe431d20640 1 -- start start 2024-04-25T11:38:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.741+0000 7fe431d20640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe42c151bc0 0x7fe42c158500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.741+0000 7fe431d20640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe42c158a40 con 0x7fe42c151fa0 2024-04-25T11:38:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.741+0000 7fe431d20640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe42c0c3930 con 0x7fe42c0722d0 2024-04-25T11:38:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.741+0000 7fe431d20640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fe42c0c3c10 con 0x7fe42c151bc0 2024-04-25T11:38:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.741+0000 7fe42affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe42c151fa0 0x7fe42c0c5510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53672/0 (socket says 172.21.15.40:53672) 2024-04-25T11:38:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.741+0000 7fe42affd640 1 -- 172.21.15.40:0/1002726737 learned_addr learned my addr 172.21.15.40:0/1002726737 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.741+0000 7fe42b7fe640 1 --2- 172.21.15.40:0/1002726737 >> v2:172.21.15.62:3300/0 conn(0x7fe42c151bc0 0x7fe42c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.742+0000 7fe42a7fc640 1 -- 172.21.15.40:0/1002726737 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349827679 0 0) 0x7fe42c158a40 con 0x7fe42c151fa0 2024-04-25T11:38:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.742+0000 7fe42a7fc640 1 -- 172.21.15.40:0/1002726737 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe414003520 con 0x7fe42c151fa0 2024-04-25T11:38:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.742+0000 7fe42a7fc640 1 -- 172.21.15.40:0/1002726737 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 458286257 0 0) 0x7fe42c0c3930 con 0x7fe42c0722d0 2024-04-25T11:38:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.742+0000 7fe42a7fc640 1 -- 172.21.15.40:0/1002726737 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe42c158a40 con 0x7fe42c0722d0 2024-04-25T11:38:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.742+0000 7fe42b7fe640 1 -- 172.21.15.40:0/1002726737 >> v1:172.21.15.62:6789/0 conn(0x7fe42c0722d0 legacy=0x7fe42c072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.742+0000 7fe42b7fe640 1 -- 172.21.15.40:0/1002726737 >> v1:172.21.15.40:6789/0 conn(0x7fe42c151fa0 legacy=0x7fe42c0c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.742+0000 7fe42b7fe640 1 -- 172.21.15.40:0/1002726737 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c0717f0 con 0x7fe42c151bc0 2024-04-25T11:38:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe42b7fe640 1 --2- 172.21.15.40:0/1002726737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe42c151bc0 0x7fe42c158500 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0097a0 tx=0x7fe41c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbfe588c7ed84c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe42a7fc640 1 -- 172.21.15.40:0/1002726737 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c037070 con 0x7fe42c151bc0 2024-04-25T11:38:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe42a7fc640 1 -- 172.21.15.40:0/1002726737 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41c032440 con 0x7fe42c151bc0 2024-04-25T11:38:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe42a7fc640 1 -- 172.21.15.40:0/1002726737 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0327f0 con 0x7fe42c151bc0 2024-04-25T11:38:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe431d20640 1 -- 172.21.15.40:0/1002726737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe42c151bc0 msgr2=0x7fe42c158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe431d20640 1 --2- 172.21.15.40:0/1002726737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe42c151bc0 0x7fe42c158500 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0097a0 tx=0x7fe41c02a230 comp rx=0 tx=0).stop 2024-04-25T11:38:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe431d20640 1 -- 172.21.15.40:0/1002726737 shutdown_connections 2024-04-25T11:38:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe431d20640 1 --2- 172.21.15.40:0/1002726737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe42c151bc0 0x7fe42c158500 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe431d20640 1 -- 172.21.15.40:0/1002726737 >> 172.21.15.40:0/1002726737 conn(0x7fe42c06b970 msgr2=0x7fe42c06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe431d20640 1 -- 172.21.15.40:0/1002726737 shutdown_connections 2024-04-25T11:38:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.743+0000 7fe431d20640 1 -- 172.21.15.40:0/1002726737 wait complete. 2024-04-25T11:38:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.744+0000 7fe431d20640 1 Processor -- start 2024-04-25T11:38:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.745+0000 7fe431d20640 1 -- start start 2024-04-25T11:38:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.745+0000 7fe431d20640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe42c0722d0 0x7fe42c1e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.745+0000 7fe431d20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c151bc0 0x7fe42c1e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.745+0000 7fe431d20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe42c151fa0 0x7fe42c1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.745+0000 7fe431d20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c0c3ef0 con 0x7fe42c151bc0 2024-04-25T11:38:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.745+0000 7fe431d20640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c071db0 con 0x7fe42c151fa0 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.745+0000 7fe431d20640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe42c06ff20 con 0x7fe42c0722d0 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c151bc0 0x7fe42c1e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42b7fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe42c0722d0 0x7fe42c1e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c151bc0 0x7fe42c1e25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51252/0 (socket says 172.21.15.40:51252) 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42affd640 1 -- 172.21.15.40:0/1874325454 learned_addr learned my addr 172.21.15.40:0/1874325454 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42bfff640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe42c151fa0 0x7fe42c1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42b7fe640 1 -- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe42c151fa0 msgr2=0x7fe42c1f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42b7fe640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe42c151fa0 0x7fe42c1f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42b7fe640 1 -- 172.21.15.40:0/1874325454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c151bc0 msgr2=0x7fe42c1e25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42b7fe640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c151bc0 0x7fe42c1e25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42b7fe640 1 -- 172.21.15.40:0/1874325454 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c152e60 con 0x7fe42c0722d0 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42affd640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c151bc0 0x7fe42c1e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe42b7fe640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe42c0722d0 0x7fe42c1e20b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe41c031000 tx=0x7fe41c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe413fff640 1 -- 172.21.15.40:0/1874325454 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c041030 con 0x7fe42c0722d0 2024-04-25T11:38:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe413fff640 1 -- 172.21.15.40:0/1874325454 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41c03d4e0 con 0x7fe42c0722d0 2024-04-25T11:38:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42c153100 con 0x7fe42c0722d0 2024-04-25T11:38:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe413fff640 1 -- 172.21.15.40:0/1874325454 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c037040 con 0x7fe42c0722d0 2024-04-25T11:38:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.746+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42c1536c0 con 0x7fe42c0722d0 2024-04-25T11:38:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.747+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42c14fdf0 con 0x7fe42c0722d0 2024-04-25T11:38:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.748+0000 7fe413fff640 1 -- 172.21.15.40:0/1874325454 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe41c044070 con 0x7fe42c0722d0 2024-04-25T11:38:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.748+0000 7fe413fff640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.748+0000 7fe413fff640 1 -- 172.21.15.40:0/1874325454 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe41c0abff0 con 0x7fe42c0722d0 2024-04-25T11:38:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.748+0000 7fe42affd640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.749+0000 7fe42affd640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe42c152d90 tx=0x7fe420009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:45.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.753+0000 7fe413fff640 1 -- 172.21.15.40:0/1874325454 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe42c14fdf0 con 0x7fe42c0722d0 2024-04-25T11:38:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.976+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe42c0c41d0 con 0x7fe42c0722d0 2024-04-25T11:38:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.977+0000 7fe413fff640 1 -- 172.21.15.40:0/1874325454 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7fe41c079eb0 con 0x7fe42c0722d0 2024-04-25T11:38:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:45.964 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:38:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe41806c8f0 msgr2=0x7fe41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe42c152d90 tx=0x7fe420009040 comp rx=0 tx=0).stop 2024-04-25T11:38:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe42c0722d0 msgr2=0x7fe42c1e20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe42c0722d0 0x7fe42c1e20b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe41c031000 tx=0x7fe41c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 shutdown_connections 2024-04-25T11:38:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe42c151fa0 0x7fe42c1f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe42c151bc0 0x7fe42c1e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 --2- 172.21.15.40:0/1874325454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe42c0722d0 0x7fe42c1e20b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 >> 172.21.15.40:0/1874325454 conn(0x7fe42c06b970 msgr2=0x7fe42c06bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 shutdown_connections 2024-04-25T11:38:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:45.979+0000 7fe431d20640 1 -- 172.21.15.40:0/1874325454 wait complete. 2024-04-25T11:38:48.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:38:49.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.104+0000 7ff901207640 1 Processor -- start 2024-04-25T11:38:49.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.104+0000 7ff901207640 1 -- start start 2024-04-25T11:38:49.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.105+0000 7ff901207640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff8fc158490 0x7ff8fc0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.105+0000 7ff901207640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.105+0000 7ff901207640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8fc0b0fa0 con 0x7ff8fc0c9cc0 2024-04-25T11:38:49.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.105+0000 7ff901207640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ff8fc0c9770 con 0x7ff8fc158490 2024-04-25T11:38:49.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.105+0000 7ff901207640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff8fc06fff0 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.105+0000 7ff8fad9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff8fc158490 0x7ff8fc0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.105+0000 7ff8fb59e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff8fc0c9cc0 0x7ff8fc15aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53680/0 (socket says 172.21.15.40:53680) 2024-04-25T11:38:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.105+0000 7ff8fb59e640 1 -- 172.21.15.40:0/1955576433 learned_addr learned my addr 172.21.15.40:0/1955576433 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.105+0000 7ff8fa59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.106+0000 7ff8f9d9b640 1 -- 172.21.15.40:0/1955576433 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1859450501 0 0) 0x7ff8fc0b0fa0 con 0x7ff8fc0c9cc0 2024-04-25T11:38:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.106+0000 7ff8fad9d640 1 -- 172.21.15.40:0/1955576433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 msgr2=0x7ff8fc0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.106+0000 7ff8fad9d640 1 --2- 172.21.15.40:0/1955576433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.106+0000 7ff8fad9d640 1 -- 172.21.15.40:0/1955576433 >> v1:172.21.15.40:6789/0 conn(0x7ff8fc0c9cc0 legacy=0x7ff8fc15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.106+0000 7ff8fad9d640 1 -- 172.21.15.40:0/1955576433 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc0702d0 con 0x7ff8fc158490 2024-04-25T11:38:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.106+0000 7ff8fa59c640 1 --2- 172.21.15.40:0/1955576433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:49.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff8fad9d640 1 --2- 172.21.15.40:0/1955576433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8fc158490 0x7ff8fc0ce260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40097a0 tx=0x7ff8f402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2776646c7d9d38a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff8f9d9b640 1 -- 172.21.15.40:0/1955576433 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4037070 con 0x7ff8fc158490 2024-04-25T11:38:49.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff8f9d9b640 1 -- 172.21.15.40:0/1955576433 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f402ae50 con 0x7ff8fc158490 2024-04-25T11:38:49.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff8f9d9b640 1 -- 172.21.15.40:0/1955576433 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4032660 con 0x7ff8fc158490 2024-04-25T11:38:49.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff901207640 1 -- 172.21.15.40:0/1955576433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8fc158490 msgr2=0x7ff8fc0ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff901207640 1 --2- 172.21.15.40:0/1955576433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8fc158490 0x7ff8fc0ce260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40097a0 tx=0x7ff8f402a100 comp rx=0 tx=0).stop 2024-04-25T11:38:49.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff901207640 1 -- 172.21.15.40:0/1955576433 shutdown_connections 2024-04-25T11:38:49.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff901207640 1 --2- 172.21.15.40:0/1955576433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff901207640 1 --2- 172.21.15.40:0/1955576433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8fc158490 0x7ff8fc0ce260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff901207640 1 -- 172.21.15.40:0/1955576433 >> 172.21.15.40:0/1955576433 conn(0x7ff8fc06bd90 msgr2=0x7ff8fc0676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:49.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff901207640 1 -- 172.21.15.40:0/1955576433 shutdown_connections 2024-04-25T11:38:49.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.107+0000 7ff901207640 1 -- 172.21.15.40:0/1955576433 wait complete. 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.109+0000 7ff901207640 1 Processor -- start 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.109+0000 7ff901207640 1 -- start start 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff901207640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc1f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff901207640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8fc0c9cc0 0x7ff8fc1f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff901207640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc158490 0x7ff8fc153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff901207640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc070780 con 0x7ff8fc158490 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff901207640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc06e970 con 0x7ff8fc0c9cc0 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff901207640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff8fc070900 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc158490 0x7ff8fc153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fad9d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc1f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fad9d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc1f3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:47414/0 (socket says 172.21.15.40:47414) 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fad9d640 1 -- 172.21.15.40:0/2577725300 learned_addr learned my addr 172.21.15.40:0/2577725300 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fa59c640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8fc0c9cc0 0x7ff8fc1f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc158490 0x7ff8fc153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51254/0 (socket says 172.21.15.40:51254) 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fad9d640 1 -- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8fc0c9cc0 msgr2=0x7ff8fc1f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fad9d640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8fc0c9cc0 0x7ff8fc1f1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fad9d640 1 -- 172.21.15.40:0/2577725300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc158490 msgr2=0x7ff8fc153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fad9d640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc158490 0x7ff8fc153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fad9d640 1 -- 172.21.15.40:0/2577725300 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc1f1790 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fb59e640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc158490 0x7ff8fc153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8fad9d640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc1f3db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40098c0 tx=0x7ff8f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.110+0000 7ff8f37fe640 1 -- 172.21.15.40:0/2577725300 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4041030 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.111+0000 7ff8f37fe640 1 -- 172.21.15.40:0/2577725300 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f402ac50 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.111+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc1f4460 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.111+0000 7ff8f37fe640 1 -- 172.21.15.40:0/2577725300 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4037040 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.111+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1f4a20 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.112+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8fc069200 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.112+0000 7ff8f37fe640 1 -- 172.21.15.40:0/2577725300 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8f4044070 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.112+0000 7ff8f37fe640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff8dc06c920 0x7ff8dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.112+0000 7ff8f37fe640 1 -- 172.21.15.40:0/2577725300 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff8f40ac140 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.112+0000 7ff8fa59c640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff8dc06c920 0x7ff8dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.113+0000 7ff8fa59c640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff8dc06c920 0x7ff8dc06ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009560 tx=0x7ff8e4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.117+0000 7ff8f37fe640 1 -- 172.21.15.40:0/2577725300 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8f4079fe0 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.342+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff8fc070b00 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.343+0000 7ff8f37fe640 1 -- 172.21.15.40:0/2577725300 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff8f407e2e0 con 0x7ff8fc0c8d50 2024-04-25T11:38:49.330 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:49.330 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff8dc06c920 msgr2=0x7ff8dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff8dc06c920 0x7ff8dc06ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009560 tx=0x7ff8e4009190 comp rx=0 tx=0).stop 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 msgr2=0x7ff8fc1f3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc1f3db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40098c0 tx=0x7ff8f40046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 shutdown_connections 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff8dc06c920 0x7ff8dc06ede0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8fc158490 0x7ff8fc153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8fc0c9cc0 0x7ff8fc1f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 --2- 172.21.15.40:0/2577725300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8fc0c8d50 0x7ff8fc1f3db0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.344+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 >> 172.21.15.40:0/2577725300 conn(0x7ff8fc06bd90 msgr2=0x7ff8fc06cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.345+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 shutdown_connections 2024-04-25T11:38:49.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.345+0000 7ff901207640 1 -- 172.21.15.40:0/2577725300 wait complete. 2024-04-25T11:38:49.342 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T11:38:49.343 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:49.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.456+0000 7f665e99b640 1 Processor -- start 2024-04-25T11:38:49.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.456+0000 7f665e99b640 1 -- start start 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.457+0000 7f665e99b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6658155140 0x7f66580d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.457+0000 7f665e99b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66580d0c00 con 0x7f66580c7ab0 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.457+0000 7f665e99b640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665815e200 con 0x7f66580cccf0 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.457+0000 7f665e99b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f665815e4e0 con 0x7f6658155140 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.457+0000 7f66577fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f66580c7ab0 0x7f66580cb210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53684/0 (socket says 172.21.15.40:53684) 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.457+0000 7f66577fe640 1 -- 172.21.15.40:0/1257196286 learned_addr learned my addr 172.21.15.40:0/1257196286 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.457+0000 7f6657fff640 1 --2- 172.21.15.40:0/1257196286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6658155140 0x7f66580d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.458+0000 7f6656ffd640 1 -- 172.21.15.40:0/1257196286 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3933758735 0 0) 0x7f665815e200 con 0x7f66580cccf0 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.458+0000 7f6656ffd640 1 -- 172.21.15.40:0/1257196286 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6640003520 con 0x7f66580cccf0 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.458+0000 7f6656ffd640 1 -- 172.21.15.40:0/1257196286 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270027534 0 0) 0x7f66580d0c00 con 0x7f66580c7ab0 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.458+0000 7f6657fff640 1 -- 172.21.15.40:0/1257196286 >> v1:172.21.15.62:6789/0 conn(0x7f66580cccf0 legacy=0x7f66580cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.458+0000 7f6657fff640 1 -- 172.21.15.40:0/1257196286 >> v1:172.21.15.40:6789/0 conn(0x7f66580c7ab0 legacy=0x7f66580cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.458+0000 7f6657fff640 1 -- 172.21.15.40:0/1257196286 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665815e7c0 con 0x7f6658155140 2024-04-25T11:38:49.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.458+0000 7f6657fff640 1 --2- 172.21.15.40:0/1257196286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6658155140 0x7f66580d03a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f66480097d0 tx=0x7f664802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1849509c89a483c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f6656ffd640 1 -- 172.21.15.40:0/1257196286 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648037070 con 0x7f6658155140 2024-04-25T11:38:49.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f6656ffd640 1 -- 172.21.15.40:0/1257196286 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6648032440 con 0x7f6658155140 2024-04-25T11:38:49.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f6656ffd640 1 -- 172.21.15.40:0/1257196286 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66480327f0 con 0x7f6658155140 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f665e99b640 1 -- 172.21.15.40:0/1257196286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6658155140 msgr2=0x7f66580d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f665e99b640 1 --2- 172.21.15.40:0/1257196286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6658155140 0x7f66580d03a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f66480097d0 tx=0x7f664802a210 comp rx=0 tx=0).stop 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f665e99b640 1 -- 172.21.15.40:0/1257196286 shutdown_connections 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f665e99b640 1 --2- 172.21.15.40:0/1257196286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6658155140 0x7f66580d03a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f665e99b640 1 -- 172.21.15.40:0/1257196286 >> 172.21.15.40:0/1257196286 conn(0x7f665806e320 msgr2=0x7f665806c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f665e99b640 1 -- 172.21.15.40:0/1257196286 shutdown_connections 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.459+0000 7f665e99b640 1 -- 172.21.15.40:0/1257196286 wait complete. 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.460+0000 7f665e99b640 1 Processor -- start 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f665e99b640 1 -- start start 2024-04-25T11:38:49.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f665e99b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66580c7ab0 0x7f6658154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f665e99b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66580cccf0 0x7f665814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f665e99b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6658155140 0x7f66581f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f665e99b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f665815f060 con 0x7f66580cccf0 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f665e99b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f665815ed80 con 0x7f6658155140 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f665e99b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f66580d08e0 con 0x7f66580c7ab0 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f66577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66580cccf0 0x7f665814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f6657fff640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66580c7ab0 0x7f6658154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f66577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66580cccf0 0x7f665814f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51256/0 (socket says 172.21.15.40:51256) 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f66577fe640 1 -- 172.21.15.40:0/2602314699 learned_addr learned my addr 172.21.15.40:0/2602314699 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.461+0000 7f665cf38640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6658155140 0x7f66581f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f66577fe640 1 -- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66580c7ab0 msgr2=0x7f6658154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f66577fe640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66580c7ab0 0x7f6658154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f66577fe640 1 -- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6658155140 msgr2=0x7f66581f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f66577fe640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6658155140 0x7f66581f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f66577fe640 1 -- 172.21.15.40:0/2602314699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665814b910 con 0x7f66580cccf0 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f6657fff640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66580c7ab0 0x7f6658154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f66577fe640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66580cccf0 0x7f665814f660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c460 tx=0x7f664c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f6654ff9640 1 -- 172.21.15.40:0/2602314699 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c018070 con 0x7f66580cccf0 2024-04-25T11:38:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f6654ff9640 1 -- 172.21.15.40:0/2602314699 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664c007500 con 0x7f66580cccf0 2024-04-25T11:38:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665814bbf0 con 0x7f66580cccf0 2024-04-25T11:38:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f6654ff9640 1 -- 172.21.15.40:0/2602314699 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c004a20 con 0x7f66580cccf0 2024-04-25T11:38:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.462+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665814c1b0 con 0x7f66580cccf0 2024-04-25T11:38:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.463+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6658069420 con 0x7f66580cccf0 2024-04-25T11:38:49.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.464+0000 7f6654ff9640 1 -- 172.21.15.40:0/2602314699 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f664c004bc0 con 0x7f66580cccf0 2024-04-25T11:38:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.464+0000 7f6654ff9640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f664406c920 0x7f664406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.464+0000 7f6654ff9640 1 -- 172.21.15.40:0/2602314699 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f664c08ae40 con 0x7f66580cccf0 2024-04-25T11:38:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.464+0000 7f6657fff640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f664406c920 0x7f664406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.465+0000 7f6657fff640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f664406c920 0x7f664406ede0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6648034000 tx=0x7f664802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.469+0000 7f6654ff9640 1 -- 172.21.15.40:0/2602314699 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6658069420 con 0x7f66580cccf0 2024-04-25T11:38:49.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.653+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6658068bb0 con 0x7f66580cccf0 2024-04-25T11:38:49.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.654+0000 7f6654ff9640 1 -- 172.21.15.40:0/2602314699 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f664c058ba0 con 0x7f66580cccf0 2024-04-25T11:38:49.641 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:49.642 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:26.664101+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f664406c920 msgr2=0x7f664406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f664406c920 0x7f664406ede0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6648034000 tx=0x7f664802e000 comp rx=0 tx=0).stop 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66580cccf0 msgr2=0x7f665814f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66580cccf0 0x7f665814f660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c460 tx=0x7f664c00c920 comp rx=0 tx=0).stop 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 shutdown_connections 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f664406c920 0x7f664406ede0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6658155140 0x7f66581f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66580cccf0 0x7f665814f660 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 --2- 172.21.15.40:0/2602314699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66580c7ab0 0x7f6658154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 >> 172.21.15.40:0/2602314699 conn(0x7f665806e320 msgr2=0x7f66580cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 shutdown_connections 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.656+0000 7f665e99b640 1 -- 172.21.15.40:0/2602314699 wait complete. 2024-04-25T11:38:49.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:38:18.972986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:38:23.454183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:38:23.968333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:38:49.655 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:38:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.772+0000 7f0d13afc640 1 Processor -- start 2024-04-25T11:38:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.773+0000 7f0d13afc640 1 -- start start 2024-04-25T11:38:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.773+0000 7f0d13afc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0d0c0cc700 0x7f0d0c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d13afc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d0c06d7b0 con 0x7f0d0c0cb990 2024-04-25T11:38:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d13afc640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d0c0c5ba0 con 0x7f0d0c155020 2024-04-25T11:38:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d13afc640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0c155400 con 0x7f0d0c0cc700 2024-04-25T11:38:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d12099640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0d0c0cb990 0x7f0d0c15e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53692/0 (socket says 172.21.15.40:53692) 2024-04-25T11:38:49.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d12099640 1 -- 172.21.15.40:0/512739194 learned_addr learned my addr 172.21.15.40:0/512739194 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d11097640 1 --2- 172.21.15.40:0/512739194 >> v2:172.21.15.62:3300/0 conn(0x7f0d0c0cc700 0x7f0d0c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d10896640 1 -- 172.21.15.40:0/512739194 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003150857 0 0) 0x7f0d0c06d7b0 con 0x7f0d0c0cb990 2024-04-25T11:38:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d10896640 1 -- 172.21.15.40:0/512739194 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cf8003520 con 0x7f0d0c0cb990 2024-04-25T11:38:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d11097640 1 -- 172.21.15.40:0/512739194 >> v1:172.21.15.62:6789/0 conn(0x7f0d0c155020 legacy=0x7f0d0c0c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d11097640 1 -- 172.21.15.40:0/512739194 >> v1:172.21.15.40:6789/0 conn(0x7f0d0c0cb990 legacy=0x7f0d0c15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d10896640 1 -- 172.21.15.40:0/512739194 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1482566281 0 0) 0x7f0d0c0c5ba0 con 0x7f0d0c155020 2024-04-25T11:38:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.774+0000 7f0d11097640 1 -- 172.21.15.40:0/512739194 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c0c5e40 con 0x7f0d0c0cc700 2024-04-25T11:38:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.775+0000 7f0d11097640 1 --2- 172.21.15.40:0/512739194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c0cc700 0x7f0d0c0cb450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0097a0 tx=0x7f0cfc02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be56c9f66e72552f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.775+0000 7f0d10896640 1 -- 172.21.15.40:0/512739194 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc037070 con 0x7f0d0c0cc700 2024-04-25T11:38:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.775+0000 7f0d10896640 1 -- 172.21.15.40:0/512739194 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cfc032440 con 0x7f0d0c0cc700 2024-04-25T11:38:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.775+0000 7f0d10896640 1 -- 172.21.15.40:0/512739194 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc032810 con 0x7f0d0c0cc700 2024-04-25T11:38:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.775+0000 7f0d13afc640 1 -- 172.21.15.40:0/512739194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c0cc700 msgr2=0x7f0d0c0cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.775+0000 7f0d13afc640 1 --2- 172.21.15.40:0/512739194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c0cc700 0x7f0d0c0cb450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0097a0 tx=0x7f0cfc02a230 comp rx=0 tx=0).stop 2024-04-25T11:38:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.775+0000 7f0d13afc640 1 -- 172.21.15.40:0/512739194 shutdown_connections 2024-04-25T11:38:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.775+0000 7f0d13afc640 1 --2- 172.21.15.40:0/512739194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c0cc700 0x7f0d0c0cb450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.775+0000 7f0d13afc640 1 -- 172.21.15.40:0/512739194 >> 172.21.15.40:0/512739194 conn(0x7f0d0c067890 msgr2=0x7f0d0c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.776+0000 7f0d13afc640 1 -- 172.21.15.40:0/512739194 shutdown_connections 2024-04-25T11:38:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.776+0000 7f0d13afc640 1 -- 172.21.15.40:0/512739194 wait complete. 2024-04-25T11:38:49.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.777+0000 7f0d13afc640 1 Processor -- start 2024-04-25T11:38:49.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.777+0000 7f0d13afc640 1 -- start start 2024-04-25T11:38:49.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.777+0000 7f0d13afc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0cb990 0x7f0d0c152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.777+0000 7f0d13afc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d0c0cc700 0x7f0d0c153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.777+0000 7f0d13afc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c155020 0x7f0d0c146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.777+0000 7f0d13afc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c15b430 con 0x7f0d0c0cb990 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.777+0000 7f0d13afc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c15b150 con 0x7f0d0c155020 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.777+0000 7f0d13afc640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0d0c15b710 con 0x7f0d0c0cc700 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.777+0000 7f0d11898640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0cb990 0x7f0d0c152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d11898640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0cb990 0x7f0d0c152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51268/0 (socket says 172.21.15.40:51268) 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d11898640 1 -- 172.21.15.40:0/656952059 learned_addr learned my addr 172.21.15.40:0/656952059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d11097640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d0c0cc700 0x7f0d0c153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d12099640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c155020 0x7f0d0c146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d11898640 1 -- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d0c0cc700 msgr2=0x7f0d0c153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d11898640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d0c0cc700 0x7f0d0c153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d11898640 1 -- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c155020 msgr2=0x7f0d0c146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d11898640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c155020 0x7f0d0c146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d11898640 1 -- 172.21.15.40:0/656952059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c15c050 con 0x7f0d0c0cb990 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d11898640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0cb990 0x7f0d0c152a10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c3e0 tx=0x7f0d0800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d027fc640 1 -- 172.21.15.40:0/656952059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08017070 con 0x7f0d0c0cb990 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d027fc640 1 -- 172.21.15.40:0/656952059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d080052d0 con 0x7f0d0c0cb990 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c1f4510 con 0x7f0d0c0cb990 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d027fc640 1 -- 172.21.15.40:0/656952059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08003830 con 0x7f0d0c0cb990 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.778+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1f4ad0 con 0x7f0d0c0cb990 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.779+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d0c06df80 con 0x7f0d0c0cb990 2024-04-25T11:38:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.780+0000 7f0d027fc640 1 -- 172.21.15.40:0/656952059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d0801a070 con 0x7f0d0c0cb990 2024-04-25T11:38:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.780+0000 7f0d027fc640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cf006c920 0x7f0cf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.780+0000 7f0d027fc640 1 -- 172.21.15.40:0/656952059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0d0808ada0 con 0x7f0d0c0cb990 2024-04-25T11:38:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.780+0000 7f0d11097640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cf006c920 0x7f0cf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.780+0000 7f0d11097640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cf006c920 0x7f0cf006ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc034000 tx=0x7f0cfc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.784+0000 7f0d027fc640 1 -- 172.21.15.40:0/656952059 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d0c06df80 con 0x7f0d0c0cb990 2024-04-25T11:38:49.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.969+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d0c0b3880 con 0x7f0d0c0cb990 2024-04-25T11:38:49.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.970+0000 7f0d027fc640 1 -- 172.21.15.40:0/656952059 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0d0c0b3880 con 0x7f0d0c0cb990 2024-04-25T11:38:49.957 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-25T11:38:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cf006c920 msgr2=0x7f0cf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cf006c920 0x7f0cf006ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc034000 tx=0x7f0cfc02e000 comp rx=0 tx=0).stop 2024-04-25T11:38:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0cb990 msgr2=0x7f0d0c152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0cb990 0x7f0d0c152a10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c3e0 tx=0x7f0d0800c8a0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 shutdown_connections 2024-04-25T11:38:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cf006c920 0x7f0cf006ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0c155020 0x7f0d0c146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d0c0cc700 0x7f0d0c153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 --2- 172.21.15.40:0/656952059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0cb990 0x7f0d0c152a10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 >> 172.21.15.40:0/656952059 conn(0x7f0d0c067890 msgr2=0x7f0d0c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 shutdown_connections 2024-04-25T11:38:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:49.972+0000 7f0d13afc640 1 -- 172.21.15.40:0/656952059 wait complete. 2024-04-25T11:38:49.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:38:50.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.086+0000 7fd9b52a7640 1 Processor -- start 2024-04-25T11:38:50.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.086+0000 7fd9b52a7640 1 -- start start 2024-04-25T11:38:50.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9b52a7640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0152f10 0x7fd9b015c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9b52a7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9b015cd70 0x7fd9b00cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9b52a7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b00c9490 con 0x7fd9b0158ad0 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9b52a7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b015c0c0 con 0x7fd9b015cd70 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9b52a7640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b0159030 con 0x7fd9b0152f10 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9ae7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9b015cd70 0x7fd9b00cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9af7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd9b0158ad0 0x7fd9b00ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53708/0 (socket says 172.21.15.40:53708) 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9af7fe640 1 -- 172.21.15.40:0/2626400141 learned_addr learned my addr 172.21.15.40:0/2626400141 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9aeffd640 1 --2- 172.21.15.40:0/2626400141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0152f10 0x7fd9b015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9ae7fc640 1 -- 172.21.15.40:0/2626400141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0152f10 msgr2=0x7fd9b015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9ae7fc640 1 --2- 172.21.15.40:0/2626400141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0152f10 0x7fd9b015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9ae7fc640 1 -- 172.21.15.40:0/2626400141 >> v1:172.21.15.40:6789/0 conn(0x7fd9b0158ad0 legacy=0x7fd9b00ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9adffb640 1 -- 172.21.15.40:0/2626400141 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4079455023 0 0) 0x7fd9b00c9490 con 0x7fd9b0158ad0 2024-04-25T11:38:50.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.087+0000 7fd9ae7fc640 1 -- 172.21.15.40:0/2626400141 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b0159310 con 0x7fd9b015cd70 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.088+0000 7fd9ae7fc640 1 --2- 172.21.15.40:0/2626400141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 0x7fd9b00cadc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40097a0 tx=0x7fd9a402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e702a19921c75c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.088+0000 7fd9adffb640 1 -- 172.21.15.40:0/2626400141 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4037070 con 0x7fd9b015cd70 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.088+0000 7fd9adffb640 1 -- 172.21.15.40:0/2626400141 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a402ae50 con 0x7fd9b015cd70 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.088+0000 7fd9adffb640 1 -- 172.21.15.40:0/2626400141 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4032660 con 0x7fd9b015cd70 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.089+0000 7fd9b52a7640 1 -- 172.21.15.40:0/2626400141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 msgr2=0x7fd9b00cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.089+0000 7fd9b52a7640 1 --2- 172.21.15.40:0/2626400141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 0x7fd9b00cadc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40097a0 tx=0x7fd9a402a100 comp rx=0 tx=0).stop 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.089+0000 7fd9b52a7640 1 -- 172.21.15.40:0/2626400141 shutdown_connections 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.089+0000 7fd9b52a7640 1 --2- 172.21.15.40:0/2626400141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 0x7fd9b00cadc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.089+0000 7fd9b52a7640 1 --2- 172.21.15.40:0/2626400141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0152f10 0x7fd9b015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.089+0000 7fd9b52a7640 1 -- 172.21.15.40:0/2626400141 >> 172.21.15.40:0/2626400141 conn(0x7fd9b006bfa0 msgr2=0x7fd9b00676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.089+0000 7fd9b52a7640 1 -- 172.21.15.40:0/2626400141 shutdown_connections 2024-04-25T11:38:50.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.089+0000 7fd9b52a7640 1 -- 172.21.15.40:0/2626400141 wait complete. 2024-04-25T11:38:50.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.090+0000 7fd9b52a7640 1 Processor -- start 2024-04-25T11:38:50.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.090+0000 7fd9b52a7640 1 -- start start 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9b52a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b0152f10 0x7fd9b014e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9b52a7640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0158ad0 0x7fd9b00716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9b52a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 0x7fd9b0156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9b52a7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b015c310 con 0x7fd9b015cd70 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9b52a7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01598d0 con 0x7fd9b0152f10 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9b52a7640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b00c5490 con 0x7fd9b0158ad0 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 0x7fd9b0156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9aeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b0152f10 0x7fd9b014e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9aeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b0152f10 0x7fd9b014e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:43190/0 (socket says 172.21.15.40:43190) 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9af7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 0x7fd9b0156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51294/0 (socket says 172.21.15.40:51294) 2024-04-25T11:38:50.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9aeffd640 1 -- 172.21.15.40:0/3192384190 learned_addr learned my addr 172.21.15.40:0/3192384190 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9ae7fc640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0158ad0 0x7fd9b00716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9af7fe640 1 -- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0158ad0 msgr2=0x7fd9b00716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9af7fe640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0158ad0 0x7fd9b00716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9af7fe640 1 -- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b0152f10 msgr2=0x7fd9b014e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9af7fe640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b0152f10 0x7fd9b014e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9af7fe640 1 -- 172.21.15.40:0/3192384190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b01f21a0 con 0x7fd9b015cd70 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9af7fe640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 0x7fd9b0156bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c6d0 tx=0x7fd9a000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9977fe640 1 -- 172.21.15.40:0/3192384190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0018070 con 0x7fd9b015cd70 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9977fe640 1 -- 172.21.15.40:0/3192384190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a0004d60 con 0x7fd9b015cd70 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.091+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b01f2480 con 0x7fd9b015cd70 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.092+0000 7fd9977fe640 1 -- 172.21.15.40:0/3192384190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000a880 con 0x7fd9b015cd70 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.092+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b01f2a40 con 0x7fd9b015cd70 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.092+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b00c3fc0 con 0x7fd9b015cd70 2024-04-25T11:38:50.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.093+0000 7fd9977fe640 1 -- 172.21.15.40:0/3192384190 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9a001b070 con 0x7fd9b015cd70 2024-04-25T11:38:50.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.093+0000 7fd9977fe640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd99c06c920 0x7fd99c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.093+0000 7fd9977fe640 1 -- 172.21.15.40:0/3192384190 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd9a008b230 con 0x7fd9b015cd70 2024-04-25T11:38:50.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.094+0000 7fd9aeffd640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd99c06c920 0x7fd99c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.094+0000 7fd9aeffd640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd99c06c920 0x7fd99c06ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009c00 tx=0x7fd9a8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.098+0000 7fd9977fe640 1 -- 172.21.15.40:0/3192384190 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9b00c3fc0 con 0x7fd9b015cd70 2024-04-25T11:38:50.254 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-04-25T11:38:50.266+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T11:38:50.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.282+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd9b00cf150 con 0x7fd9b015cd70 2024-04-25T11:38:50.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.282+0000 7fd9977fe640 1 -- 172.21.15.40:0/3192384190 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd9b00cf150 con 0x7fd9b015cd70 2024-04-25T11:38:50.269 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-25T11:38:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd99c06c920 msgr2=0x7fd99c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd99c06c920 0x7fd99c06ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009c00 tx=0x7fd9a8009190 comp rx=0 tx=0).stop 2024-04-25T11:38:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 msgr2=0x7fd9b0156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 0x7fd9b0156bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000c6d0 tx=0x7fd9a000cb90 comp rx=0 tx=0).stop 2024-04-25T11:38:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 shutdown_connections 2024-04-25T11:38:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd99c06c920 0x7fd99c06ede0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b015cd70 0x7fd9b0156bd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9b0158ad0 0x7fd9b00716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 --2- 172.21.15.40:0/3192384190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b0152f10 0x7fd9b014e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 >> 172.21.15.40:0/3192384190 conn(0x7fd9b006bfa0 msgr2=0x7fd9b00c4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 shutdown_connections 2024-04-25T11:38:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.284+0000 7fd9b52a7640 1 -- 172.21.15.40:0/3192384190 wait complete. 2024-04-25T11:38:50.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:38:50.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.398+0000 7f3736260640 1 Processor -- start 2024-04-25T11:38:50.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.398+0000 7f3736260640 1 -- start start 2024-04-25T11:38:50.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f3736260640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 0x7f37300cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f3736260640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f373015ae70 con 0x7f37300cb830 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f3736260640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f373015b150 con 0x7f37300cc510 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f3736260640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f373015b430 con 0x7f3730154e80 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f372effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f37300cb830 0x7f37300cd370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53716/0 (socket says 172.21.15.40:53716) 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f372effd640 1 -- 172.21.15.40:0/1503277921 learned_addr learned my addr 172.21.15.40:0/1503277921 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f372f7fe640 1 --2- 172.21.15.40:0/1503277921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 0x7f37300cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f372e7fc640 1 -- 172.21.15.40:0/1503277921 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775041885 0 0) 0x7f373015ae70 con 0x7f37300cb830 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f372f7fe640 1 -- 172.21.15.40:0/1503277921 >> v1:172.21.15.62:6789/0 conn(0x7f37300cc510 legacy=0x7f373006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f372f7fe640 1 -- 172.21.15.40:0/1503277921 >> v1:172.21.15.40:6789/0 conn(0x7f37300cb830 legacy=0x7f37300cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.399+0000 7f372f7fe640 1 -- 172.21.15.40:0/1503277921 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373015b710 con 0x7f3730154e80 2024-04-25T11:38:50.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.400+0000 7f372f7fe640 1 --2- 172.21.15.40:0/1503277921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 0x7f37300cb2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3720009750 tx=0x7f372002a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=128e18a313aedccd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.400+0000 7f372e7fc640 1 -- 172.21.15.40:0/1503277921 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720037070 con 0x7f3730154e80 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.400+0000 7f372e7fc640 1 -- 172.21.15.40:0/1503277921 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372002adb0 con 0x7f3730154e80 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.400+0000 7f372e7fc640 1 -- 172.21.15.40:0/1503277921 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720032660 con 0x7f3730154e80 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.401+0000 7f3736260640 1 -- 172.21.15.40:0/1503277921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 msgr2=0x7f37300cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.401+0000 7f3736260640 1 --2- 172.21.15.40:0/1503277921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 0x7f37300cb2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3720009750 tx=0x7f372002a0a0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.401+0000 7f3736260640 1 -- 172.21.15.40:0/1503277921 shutdown_connections 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.401+0000 7f3736260640 1 --2- 172.21.15.40:0/1503277921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 0x7f37300cb2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.401+0000 7f3736260640 1 -- 172.21.15.40:0/1503277921 >> 172.21.15.40:0/1503277921 conn(0x7f3730067890 msgr2=0x7f3730067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.401+0000 7f3736260640 1 -- 172.21.15.40:0/1503277921 shutdown_connections 2024-04-25T11:38:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.401+0000 7f3736260640 1 -- 172.21.15.40:0/1503277921 wait complete. 2024-04-25T11:38:50.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.402+0000 7f3736260640 1 Processor -- start 2024-04-25T11:38:50.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.402+0000 7f3736260640 1 -- start start 2024-04-25T11:38:50.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f3736260640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37300cb830 0x7f373015be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f3736260640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37300cc510 0x7f373015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f3736260640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 0x7f37301f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f3736260640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37301553e0 con 0x7f37300cb830 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f3736260640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3730155260 con 0x7f37300cc510 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f3736260640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f37300cda80 con 0x7f3730154e80 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37300cb830 0x7f373015be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37300cb830 0x7f373015be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51306/0 (socket says 172.21.15.40:51306) 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372f7fe640 1 -- 172.21.15.40:0/2818041385 learned_addr learned my addr 172.21.15.40:0/2818041385 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372effd640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37300cc510 0x7f373015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372ffff640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 0x7f37301f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372f7fe640 1 -- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 msgr2=0x7f37301f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372f7fe640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 0x7f37301f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372f7fe640 1 -- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37300cc510 msgr2=0x7f373015db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372f7fe640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37300cc510 0x7f373015db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372f7fe640 1 -- 172.21.15.40:0/2818041385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373015e8d0 con 0x7f37300cb830 2024-04-25T11:38:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f372f7fe640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37300cb830 0x7f373015be40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3720009870 tx=0x7f37200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.403+0000 7f3717fff640 1 -- 172.21.15.40:0/2818041385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720037040 con 0x7f37300cb830 2024-04-25T11:38:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.404+0000 7f3717fff640 1 -- 172.21.15.40:0/2818041385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3720004230 con 0x7f37300cb830 2024-04-25T11:38:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.404+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373015ebb0 con 0x7f37300cb830 2024-04-25T11:38:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.404+0000 7f3717fff640 1 -- 172.21.15.40:0/2818041385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720003610 con 0x7f37300cb830 2024-04-25T11:38:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.404+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37301f0ba0 con 0x7f37300cb830 2024-04-25T11:38:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.404+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37301f06b0 con 0x7f37300cb830 2024-04-25T11:38:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.405+0000 7f3717fff640 1 -- 172.21.15.40:0/2818041385 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f372002ac30 con 0x7f37300cb830 2024-04-25T11:38:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.405+0000 7f3717fff640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f371c06c920 0x7f371c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.405+0000 7f3717fff640 1 -- 172.21.15.40:0/2818041385 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f37200ac480 con 0x7f37300cb830 2024-04-25T11:38:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.406+0000 7f372effd640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f371c06c920 0x7f371c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.406+0000 7f372effd640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f371c06c920 0x7f371c06ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3724007f70 tx=0x7f3724007f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.410+0000 7f3717fff640 1 -- 172.21.15.40:0/2818041385 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f372007a1d0 con 0x7f37300cb830 2024-04-25T11:38:50.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.598+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3730068bb0 con 0x7f37300cb830 2024-04-25T11:38:50.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.598+0000 7f3717fff640 1 -- 172.21.15.40:0/2818041385 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f372007e4d0 con 0x7f37300cb830 2024-04-25T11:38:50.586 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f371c06c920 msgr2=0x7f371c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f371c06c920 0x7f371c06ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3724007f70 tx=0x7f3724007f00 comp rx=0 tx=0).stop 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37300cb830 msgr2=0x7f373015be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37300cb830 0x7f373015be40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3720009870 tx=0x7f37200046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 shutdown_connections 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f371c06c920 0x7f371c06ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3730154e80 0x7f37301f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37300cc510 0x7f373015db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 --2- 172.21.15.40:0/2818041385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37300cb830 0x7f373015be40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 >> 172.21.15.40:0/2818041385 conn(0x7f3730067890 msgr2=0x7f373006c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 shutdown_connections 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.600+0000 7f3736260640 1 -- 172.21.15.40:0/2818041385 wait complete. 2024-04-25T11:38:50.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.711+0000 7f5979d0f640 1 Processor -- start 2024-04-25T11:38:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.711+0000 7f5979d0f640 1 -- start start 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.712+0000 7f5979d0f640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5974156130 0x7f59740c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.712+0000 7f5979d0f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59740c9640 con 0x7f597415c0f0 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.712+0000 7f5979d0f640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59740ca2e0 con 0x7f597406fee0 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.712+0000 7f5979d0f640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f59740ca5c0 con 0x7f5974156130 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.712+0000 7f5972ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f597415c0f0 0x7f59740c8d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53730/0 (socket says 172.21.15.40:53730) 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.712+0000 7f5972ffd640 1 -- 172.21.15.40:0/972489947 learned_addr learned my addr 172.21.15.40:0/972489947 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.712+0000 7f5973fff640 1 --1- 172.21.15.40:0/972489947 >> v1:172.21.15.62:6789/0 conn(0x7f597406fee0 0x7f5974163420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:45382/0 (socket says 172.21.15.40:45382) 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.712+0000 7f59737fe640 1 --2- 172.21.15.40:0/972489947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5974156130 0x7f59740c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.713+0000 7f59737fe640 1 -- 172.21.15.40:0/972489947 >> v1:172.21.15.62:6789/0 conn(0x7f597406fee0 legacy=0x7f5974163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.713+0000 7f59737fe640 1 -- 172.21.15.40:0/972489947 >> v1:172.21.15.40:6789/0 conn(0x7f597415c0f0 legacy=0x7f59740c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.713+0000 7f59727fc640 1 -- 172.21.15.40:0/972489947 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446624613 0 0) 0x7f59740ca2e0 con 0x7f597406fee0 2024-04-25T11:38:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.713+0000 7f59737fe640 1 -- 172.21.15.40:0/972489947 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59740ca8a0 con 0x7f5974156130 2024-04-25T11:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.713+0000 7f59737fe640 1 --2- 172.21.15.40:0/972489947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5974156130 0x7f59740c51c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59640097d0 tx=0x7f596402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56226ffe0544b85f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.713+0000 7f59727fc640 1 -- 172.21.15.40:0/972489947 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964037070 con 0x7f5974156130 2024-04-25T11:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.714+0000 7f59727fc640 1 -- 172.21.15.40:0/972489947 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596402ae30 con 0x7f5974156130 2024-04-25T11:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.714+0000 7f59727fc640 1 -- 172.21.15.40:0/972489947 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964032640 con 0x7f5974156130 2024-04-25T11:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.714+0000 7f5979d0f640 1 -- 172.21.15.40:0/972489947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5974156130 msgr2=0x7f59740c51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.714+0000 7f5979d0f640 1 --2- 172.21.15.40:0/972489947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5974156130 0x7f59740c51c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59640097d0 tx=0x7f596402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.714+0000 7f5979d0f640 1 -- 172.21.15.40:0/972489947 shutdown_connections 2024-04-25T11:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.714+0000 7f5979d0f640 1 --2- 172.21.15.40:0/972489947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5974156130 0x7f59740c51c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.714+0000 7f5979d0f640 1 -- 172.21.15.40:0/972489947 >> 172.21.15.40:0/972489947 conn(0x7f597406bd90 msgr2=0x7f59740676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.714+0000 7f5979d0f640 1 -- 172.21.15.40:0/972489947 shutdown_connections 2024-04-25T11:38:50.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.714+0000 7f5979d0f640 1 -- 172.21.15.40:0/972489947 wait complete. 2024-04-25T11:38:50.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.715+0000 7f5979d0f640 1 Processor -- start 2024-04-25T11:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.715+0000 7f5979d0f640 1 -- start start 2024-04-25T11:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f5979d0f640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406fee0 0x7f59741f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f5979d0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5974156130 0x7f5974155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f5979d0f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597415c0f0 0x7f5974157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f5979d0f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59740cbb90 con 0x7f597415c0f0 2024-04-25T11:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f5979d0f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f59740ce640 con 0x7f5974156130 2024-04-25T11:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f5979d0f640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5974156510 con 0x7f597406fee0 2024-04-25T11:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f5973fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597415c0f0 0x7f5974157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f59737fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406fee0 0x7f59741f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f59737fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406fee0 0x7f59741f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:47496/0 (socket says 172.21.15.40:47496) 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f59737fe640 1 -- 172.21.15.40:0/1004055595 learned_addr learned my addr 172.21.15.40:0/1004055595 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.716+0000 7f5972ffd640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5974156130 0x7f5974155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f59737fe640 1 -- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5974156130 msgr2=0x7f5974155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f59737fe640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5974156130 0x7f5974155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f59737fe640 1 -- 172.21.15.40:0/1004055595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597415c0f0 msgr2=0x7f5974157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f59737fe640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597415c0f0 0x7f5974157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f59737fe640 1 -- 172.21.15.40:0/1004055595 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5974156da0 con 0x7f597406fee0 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f5973fff640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597415c0f0 0x7f5974157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f59737fe640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406fee0 0x7f59741f2760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5964000c00 tx=0x7f59640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f595bfff640 1 -- 172.21.15.40:0/1004055595 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964037040 con 0x7f597406fee0 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f595bfff640 1 -- 172.21.15.40:0/1004055595 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5964004030 con 0x7f597406fee0 2024-04-25T11:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f595bfff640 1 -- 172.21.15.40:0/1004055595 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964004420 con 0x7f597406fee0 2024-04-25T11:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5974159ef0 con 0x7f597406fee0 2024-04-25T11:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.717+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f597415a4b0 con 0x7f597406fee0 2024-04-25T11:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.718+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59740ca2e0 con 0x7f597406fee0 2024-04-25T11:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.718+0000 7f595bfff640 1 -- 172.21.15.40:0/1004055595 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f596403a040 con 0x7f597406fee0 2024-04-25T11:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.719+0000 7f595bfff640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f596006c8f0 0x7f596006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.719+0000 7f595bfff640 1 -- 172.21.15.40:0/1004055595 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f59640ac1b0 con 0x7f597406fee0 2024-04-25T11:38:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.719+0000 7f5972ffd640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f596006c8f0 0x7f596006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.719+0000 7f5972ffd640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f596006c8f0 0x7f596006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59741ef910 tx=0x7f5968007e20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.723+0000 7f595bfff640 1 -- 172.21.15.40:0/1004055595 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59740ca2e0 con 0x7f597406fee0 2024-04-25T11:38:50.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.944+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5974069200 con 0x7f597406fee0 2024-04-25T11:38:50.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.944+0000 7f595bfff640 1 -- 172.21.15.40:0/1004055595 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6900 (secure 0 0 0) 0x7f5964042020 con 0x7f597406fee0 2024-04-25T11:38:50.932 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:50.932 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1073487120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1073487120},{"type":"v1","addr":"172.21.15.40:6837","nonce":1073487120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:38:25.023156+0000","modified":"2024-04-25T11:38:48.036680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4300,"mds_1":4294,"mds_2":4279,"mds_3":4260,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"d","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.62:6838/3333589902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3333589902},{"type":"v1","addr":"172.21.15.62:6838","nonce":3333589902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4260":{"gid":4260,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4104143657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4104143657},{"type":"v1","addr":"172.21.15.40:6835","nonce":4104143657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6836/535060457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":535060457},{"type":"v1","addr":"172.21.15.40:6836","nonce":535060457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4294":{"gid":4294,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6835/3758619338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3758619338},{"type":"v1","addr":"172.21.15.62:6835","nonce":3758619338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/2121129816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6837","nonce":2121129816},{"type":"v1","addr":"172.21.15.62:6839","nonce":2121129816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4294,4279,4260]},"id":1}]} 2024-04-25T11:38:50.932 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-04-25T11:38:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f596006c8f0 msgr2=0x7f596006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f596006c8f0 0x7f596006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59741ef910 tx=0x7f5968007e20 comp rx=0 tx=0).stop 2024-04-25T11:38:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406fee0 msgr2=0x7f59741f2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406fee0 0x7f59741f2760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5964000c00 tx=0x7f59640046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 shutdown_connections 2024-04-25T11:38:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f596006c8f0 0x7f596006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597415c0f0 0x7f5974157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5974156130 0x7f5974155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 --2- 172.21.15.40:0/1004055595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f597406fee0 0x7f59741f2760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.946+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 >> 172.21.15.40:0/1004055595 conn(0x7f597406bd90 msgr2=0x7f5974154660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.947+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 shutdown_connections 2024-04-25T11:38:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:50.947+0000 7f5979d0f640 1 -- 172.21.15.40:0/1004055595 wait complete. 2024-04-25T11:38:50.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:51.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.058+0000 7f2b46f5d640 1 Processor -- start 2024-04-25T11:38:51.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.058+0000 7f2b46f5d640 1 -- start start 2024-04-25T11:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b46f5d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 0x7f2b400c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b46f5d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b400c8690 0x7f2b400c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b46f5d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b400c9060 con 0x7f2b400b6210 2024-04-25T11:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b46f5d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4015eb20 con 0x7f2b400c8690 2024-04-25T11:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b46f5d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4015f060 con 0x7f2b40154f50 2024-04-25T11:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b454fa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b400c8690 0x7f2b400c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b3ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2b400b6210 0x7f2b400cb2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53736/0 (socket says 172.21.15.40:53736) 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b454fa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b400c8690 0x7f2b400c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51328/0 (socket says 172.21.15.40:51328) 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b454fa640 1 -- 172.21.15.40:0/3460878224 learned_addr learned my addr 172.21.15.40:0/3460878224 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b44cf9640 1 --2- 172.21.15.40:0/3460878224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 0x7f2b400c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b454fa640 1 -- 172.21.15.40:0/3460878224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 msgr2=0x7f2b400c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b454fa640 1 --2- 172.21.15.40:0/3460878224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 0x7f2b400c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b454fa640 1 -- 172.21.15.40:0/3460878224 >> v1:172.21.15.40:6789/0 conn(0x7f2b400b6210 legacy=0x7f2b400cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b3f7fe640 1 -- 172.21.15.40:0/3460878224 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 987690993 0 0) 0x7f2b400c9060 con 0x7f2b400b6210 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.059+0000 7f2b454fa640 1 -- 172.21.15.40:0/3460878224 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b400d1040 con 0x7f2b400c8690 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.060+0000 7f2b454fa640 1 --2- 172.21.15.40:0/3460878224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b400c8690 0x7f2b400c8b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009e50 tx=0x7f2b3802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e042a057eb52d7f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.060+0000 7f2b3f7fe640 1 -- 172.21.15.40:0/3460878224 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38036070 con 0x7f2b400c8690 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.060+0000 7f2b3f7fe640 1 -- 172.21.15.40:0/3460878224 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b38031600 con 0x7f2b400c8690 2024-04-25T11:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.060+0000 7f2b3f7fe640 1 -- 172.21.15.40:0/3460878224 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380319d0 con 0x7f2b400c8690 2024-04-25T11:38:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.061+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3460878224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b400c8690 msgr2=0x7f2b400c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.061+0000 7f2b46f5d640 1 --2- 172.21.15.40:0/3460878224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b400c8690 0x7f2b400c8b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009e50 tx=0x7f2b3802a680 comp rx=0 tx=0).stop 2024-04-25T11:38:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.061+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3460878224 shutdown_connections 2024-04-25T11:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.061+0000 7f2b46f5d640 1 --2- 172.21.15.40:0/3460878224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b400c8690 0x7f2b400c8b20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.061+0000 7f2b46f5d640 1 --2- 172.21.15.40:0/3460878224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 0x7f2b400c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.061+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3460878224 >> 172.21.15.40:0/3460878224 conn(0x7f2b400d0440 msgr2=0x7f2b400d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.061+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3460878224 shutdown_connections 2024-04-25T11:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.061+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3460878224 wait complete. 2024-04-25T11:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.062+0000 7f2b46f5d640 1 Processor -- start 2024-04-25T11:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.062+0000 7f2b46f5d640 1 -- start start 2024-04-25T11:38:51.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b46f5d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b400c8690 0x7f2b40156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b46f5d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 0x7f2b4014e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b46f5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b401f1990 0x7f2b401f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b46f5d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4015ed80 con 0x7f2b401f1990 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b46f5d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b400cc150 con 0x7f2b400c8690 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b46f5d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2b400cc3f0 con 0x7f2b40154f50 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b454fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b401f1990 0x7f2b401f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b454fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b401f1990 0x7f2b401f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51332/0 (socket says 172.21.15.40:51332) 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b454fa640 1 -- 172.21.15.40:0/3078652364 learned_addr learned my addr 172.21.15.40:0/3078652364 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b44cf9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b400c8690 0x7f2b40156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b3ffff640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 0x7f2b4014e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b454fa640 1 -- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 msgr2=0x7f2b4014e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b454fa640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 0x7f2b4014e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b454fa640 1 -- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b400c8690 msgr2=0x7f2b40156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b454fa640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b400c8690 0x7f2b40156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b454fa640 1 -- 172.21.15.40:0/3078652364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b401573d0 con 0x7f2b401f1990 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b3ffff640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 0x7f2b4014e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b454fa640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b401f1990 0x7f2b401f3a50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009f70 tx=0x7f2b38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.063+0000 7f2b3d7fa640 1 -- 172.21.15.40:0/3078652364 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3803f020 con 0x7f2b401f1990 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.064+0000 7f2b3d7fa640 1 -- 172.21.15.40:0/3078652364 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b38037040 con 0x7f2b401f1990 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.064+0000 7f2b3d7fa640 1 -- 172.21.15.40:0/3078652364 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38036070 con 0x7f2b401f1990 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.064+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b401f4630 con 0x7f2b401f1990 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.064+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b401563b0 con 0x7f2b401f1990 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.065+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b400caa90 con 0x7f2b401f1990 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.065+0000 7f2b3d7fa640 1 -- 172.21.15.40:0/3078652364 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b38042020 con 0x7f2b401f1990 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.065+0000 7f2b3d7fa640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2b2406c920 0x7f2b2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.066+0000 7f2b3d7fa640 1 -- 172.21.15.40:0/3078652364 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2b380abd30 con 0x7f2b401f1990 2024-04-25T11:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.066+0000 7f2b44cf9640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2b2406c920 0x7f2b2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.066+0000 7f2b44cf9640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2b2406c920 0x7f2b2406ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009aa0 tx=0x7f2b34009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.070+0000 7f2b3d7fa640 1 -- 172.21.15.40:0/3078652364 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b400caa90 con 0x7f2b401f1990 2024-04-25T11:38:51.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.255+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b40068bb0 con 0x7f2b401f1990 2024-04-25T11:38:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.256+0000 7f2b3d7fa640 1 -- 172.21.15.40:0/3078652364 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f2b38049030 con 0x7f2b401f1990 2024-04-25T11:38:51.244 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:51.244 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:26.664101+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2b2406c920 msgr2=0x7f2b2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2b2406c920 0x7f2b2406ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009aa0 tx=0x7f2b34009190 comp rx=0 tx=0).stop 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b401f1990 msgr2=0x7f2b401f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b401f1990 0x7f2b401f3a50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009f70 tx=0x7f2b38004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 shutdown_connections 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2b2406c920 0x7f2b2406ede0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b401f1990 0x7f2b401f3a50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b40154f50 0x7f2b4014e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 --2- 172.21.15.40:0/3078652364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b400c8690 0x7f2b40156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 >> 172.21.15.40:0/3078652364 conn(0x7f2b400d0440 msgr2=0x7f2b40152770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 shutdown_connections 2024-04-25T11:38:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.258+0000 7f2b46f5d640 1 -- 172.21.15.40:0/3078652364 wait complete. 2024-04-25T11:38:51.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:38:18.972986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:38:23.454183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:38:23.968333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:38:51.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:38:51.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.373+0000 7fefe6cef640 1 Processor -- start 2024-04-25T11:38:51.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.373+0000 7fefe6cef640 1 -- start start 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe6cef640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fefe0155230 0x7fefe00cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe6cef640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe006ce40 0x7fefe00cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe6cef640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe00ccdb0 con 0x7fefe00c56e0 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe6cef640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fefe00cb950 con 0x7fefe0155230 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe6cef640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fefe00cc7f0 con 0x7fefe006ce40 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefdffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fefe00c56e0 0x7fefe015b800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53750/0 (socket says 172.21.15.40:53750) 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefdffff640 1 -- 172.21.15.40:0/2925247790 learned_addr learned my addr 172.21.15.40:0/2925247790 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe528c640 1 --2- 172.21.15.40:0/2925247790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe006ce40 0x7fefe00cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe4a8b640 1 --2- 172.21.15.40:0/2925247790 >> v2:172.21.15.62:3300/0 conn(0x7fefe0155230 0x7fefe00cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe528c640 1 -- 172.21.15.40:0/2925247790 >> v2:172.21.15.62:3300/0 conn(0x7fefe0155230 msgr2=0x7fefe00cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe528c640 1 --2- 172.21.15.40:0/2925247790 >> v2:172.21.15.62:3300/0 conn(0x7fefe0155230 0x7fefe00cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe528c640 1 -- 172.21.15.40:0/2925247790 >> v1:172.21.15.40:6789/0 conn(0x7fefe00c56e0 legacy=0x7fefe015b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefdf7fe640 1 -- 172.21.15.40:0/2925247790 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4193708990 0 0) 0x7fefcc001110 con 0x7fefe00c56e0 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.374+0000 7fefe528c640 1 -- 172.21.15.40:0/2925247790 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe00ccad0 con 0x7fefe006ce40 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.375+0000 7fefe528c640 1 --2- 172.21.15.40:0/2925247790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe006ce40 0x7fefe00cd3d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009d90 tx=0x7fefd802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108eb49b4fac1a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.375+0000 7fefdf7fe640 1 -- 172.21.15.40:0/2925247790 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8007740 con 0x7fefe006ce40 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.375+0000 7fefdf7fe640 1 -- 172.21.15.40:0/2925247790 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd80078e0 con 0x7fefe006ce40 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.375+0000 7fefdf7fe640 1 -- 172.21.15.40:0/2925247790 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8007cb0 con 0x7fefe006ce40 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.375+0000 7fefe6cef640 1 -- 172.21.15.40:0/2925247790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe006ce40 msgr2=0x7fefe00cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.375+0000 7fefe6cef640 1 --2- 172.21.15.40:0/2925247790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe006ce40 0x7fefe00cd3d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009d90 tx=0x7fefd802a660 comp rx=0 tx=0).stop 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.376+0000 7fefe6cef640 1 -- 172.21.15.40:0/2925247790 shutdown_connections 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.376+0000 7fefe6cef640 1 --2- 172.21.15.40:0/2925247790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe006ce40 0x7fefe00cd3d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.376+0000 7fefe6cef640 1 --2- 172.21.15.40:0/2925247790 >> v2:172.21.15.62:3300/0 conn(0x7fefe0155230 0x7fefe00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.376+0000 7fefe6cef640 1 -- 172.21.15.40:0/2925247790 >> 172.21.15.40:0/2925247790 conn(0x7fefe0067890 msgr2=0x7fefe0067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.376+0000 7fefe6cef640 1 -- 172.21.15.40:0/2925247790 shutdown_connections 2024-04-25T11:38:51.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.376+0000 7fefe6cef640 1 -- 172.21.15.40:0/2925247790 wait complete. 2024-04-25T11:38:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.377+0000 7fefe6cef640 1 Processor -- start 2024-04-25T11:38:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.377+0000 7fefe6cef640 1 -- start start 2024-04-25T11:38:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.377+0000 7fefe6cef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefe006ce40 0x7fefe014d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe6cef640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe00c56e0 0x7fefe0153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe6cef640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155230 0x7fefe0153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe6cef640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fefe006d2f0 con 0x7fefe006ce40 2024-04-25T11:38:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe6cef640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefe006c920 con 0x7fefe0155230 2024-04-25T11:38:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe6cef640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fefe006d470 con 0x7fefe00c56e0 2024-04-25T11:38:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe4a8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefe006ce40 0x7fefe014d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe4a8b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefe006ce40 0x7fefe014d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51334/0 (socket says 172.21.15.40:51334) 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe4a8b640 1 -- 172.21.15.40:0/3283887040 learned_addr learned my addr 172.21.15.40:0/3283887040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe528c640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155230 0x7fefe0153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefdffff640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe00c56e0 0x7fefe0153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe4a8b640 1 -- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe00c56e0 msgr2=0x7fefe0153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe4a8b640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe00c56e0 0x7fefe0153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe4a8b640 1 -- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155230 msgr2=0x7fefe0153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe4a8b640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155230 0x7fefe0153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe4a8b640 1 -- 172.21.15.40:0/3283887040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe014d5b0 con 0x7fefe006ce40 2024-04-25T11:38:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe4a8b640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefe006ce40 0x7fefe014d070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fefd400e9e0 tx=0x7fefd400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefdd7fa640 1 -- 172.21.15.40:0/3283887040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4017070 con 0x7fefe006ce40 2024-04-25T11:38:51.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefdd7fa640 1 -- 172.21.15.40:0/3283887040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd400c970 con 0x7fefe006ce40 2024-04-25T11:38:51.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe01f3fc0 con 0x7fefe006ce40 2024-04-25T11:38:51.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefdd7fa640 1 -- 172.21.15.40:0/3283887040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd400cd60 con 0x7fefe006ce40 2024-04-25T11:38:51.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.378+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe01f4580 con 0x7fefe006ce40 2024-04-25T11:38:51.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.379+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe0068bb0 con 0x7fefe006ce40 2024-04-25T11:38:51.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.380+0000 7fefdd7fa640 1 -- 172.21.15.40:0/3283887040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefd401a070 con 0x7fefe006ce40 2024-04-25T11:38:51.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.380+0000 7fefdd7fa640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fefc406c920 0x7fefc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.381+0000 7fefdd7fa640 1 -- 172.21.15.40:0/3283887040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fefd408bd80 con 0x7fefe006ce40 2024-04-25T11:38:51.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.381+0000 7fefdffff640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fefc406c920 0x7fefc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.381+0000 7fefdffff640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fefc406c920 0x7fefc406ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fefe015b4a0 tx=0x7fefcc007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.385+0000 7fefdd7fa640 1 -- 172.21.15.40:0/3283887040 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefd4059b60 con 0x7fefe006ce40 2024-04-25T11:38:51.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.569+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fefe00b3070 con 0x7fefe006ce40 2024-04-25T11:38:51.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.569+0000 7fefdd7fa640 1 -- 172.21.15.40:0/3283887040 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fefd405de60 con 0x7fefe006ce40 2024-04-25T11:38:51.557 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-25T11:38:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.571+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fefc406c920 msgr2=0x7fefc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.571+0000 7fefe6cef640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fefc406c920 0x7fefc406ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fefe015b4a0 tx=0x7fefcc007de0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.571+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefe006ce40 msgr2=0x7fefe014d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.571+0000 7fefe6cef640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefe006ce40 0x7fefe014d070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fefd400e9e0 tx=0x7fefd400eea0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.572+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 shutdown_connections 2024-04-25T11:38:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.572+0000 7fefe6cef640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fefc406c920 0x7fefc406ede0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.572+0000 7fefe6cef640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155230 0x7fefe0153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.572+0000 7fefe6cef640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe00c56e0 0x7fefe0153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.572+0000 7fefe6cef640 1 --2- 172.21.15.40:0/3283887040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefe006ce40 0x7fefe014d070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.572+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 >> 172.21.15.40:0/3283887040 conn(0x7fefe0067890 msgr2=0x7fefe00c68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.572+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 shutdown_connections 2024-04-25T11:38:51.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.572+0000 7fefe6cef640 1 -- 172.21.15.40:0/3283887040 wait complete. 2024-04-25T11:38:51.569 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:38:51.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.686+0000 7f5559efd640 1 Processor -- start 2024-04-25T11:38:51.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.686+0000 7f5559efd640 1 -- start start 2024-04-25T11:38:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f5559efd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5554155070 0x7f555415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f5559efd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55540cbe40 con 0x7f555415b8e0 2024-04-25T11:38:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f5559efd640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55540cb880 con 0x7f55540cc820 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f5559efd640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f555415be20 con 0x7f5554155070 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f5552ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f555415b8e0 0x7f55540d04f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53760/0 (socket says 172.21.15.40:53760) 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f5552ffd640 1 -- 172.21.15.40:0/3717055091 learned_addr learned my addr 172.21.15.40:0/3717055091 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f55537fe640 1 --2- 172.21.15.40:0/3717055091 >> v2:172.21.15.62:3300/0 conn(0x7f5554155070 0x7f555415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f55527fc640 1 -- 172.21.15.40:0/3717055091 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3885377979 0 0) 0x7f55540cbe40 con 0x7f555415b8e0 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f55527fc640 1 -- 172.21.15.40:0/3717055091 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f553c003520 con 0x7f555415b8e0 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f55527fc640 1 -- 172.21.15.40:0/3717055091 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1102729957 0 0) 0x7f55540cb880 con 0x7f55540cc820 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f55537fe640 1 -- 172.21.15.40:0/3717055091 >> v1:172.21.15.62:6789/0 conn(0x7f55540cc820 legacy=0x7f55541657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f55537fe640 1 -- 172.21.15.40:0/3717055091 >> v1:172.21.15.40:6789/0 conn(0x7f555415b8e0 legacy=0x7f55540d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.687+0000 7f55537fe640 1 -- 172.21.15.40:0/3717055091 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55540cbb60 con 0x7f5554155070 2024-04-25T11:38:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.688+0000 7f55537fe640 1 --2- 172.21.15.40:0/3717055091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5554155070 0x7f555415ed60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f55440097a0 tx=0x7f554402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f889fe70752116b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.688+0000 7f55527fc640 1 -- 172.21.15.40:0/3717055091 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544037070 con 0x7f5554155070 2024-04-25T11:38:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.688+0000 7f55527fc640 1 -- 172.21.15.40:0/3717055091 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5544032440 con 0x7f5554155070 2024-04-25T11:38:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.688+0000 7f55527fc640 1 -- 172.21.15.40:0/3717055091 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544032810 con 0x7f5554155070 2024-04-25T11:38:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.689+0000 7f5559efd640 1 -- 172.21.15.40:0/3717055091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5554155070 msgr2=0x7f555415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.689+0000 7f5559efd640 1 --2- 172.21.15.40:0/3717055091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5554155070 0x7f555415ed60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f55440097a0 tx=0x7f554402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.689+0000 7f5559efd640 1 -- 172.21.15.40:0/3717055091 shutdown_connections 2024-04-25T11:38:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.689+0000 7f5559efd640 1 --2- 172.21.15.40:0/3717055091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5554155070 0x7f555415ed60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.689+0000 7f5559efd640 1 -- 172.21.15.40:0/3717055091 >> 172.21.15.40:0/3717055091 conn(0x7f555406c680 msgr2=0x7f555406e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.689+0000 7f5559efd640 1 -- 172.21.15.40:0/3717055091 shutdown_connections 2024-04-25T11:38:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.689+0000 7f5559efd640 1 -- 172.21.15.40:0/3717055091 wait complete. 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.690+0000 7f5559efd640 1 Processor -- start 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.690+0000 7f5559efd640 1 -- start start 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5559efd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55540cc820 0x7f55540c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5559efd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5554155070 0x7f55540c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5559efd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555415b8e0 0x7f555415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5559efd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55540cb4e0 con 0x7f555415b8e0 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5559efd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f555415e840 con 0x7f55540cc820 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5559efd640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f555415f2a0 con 0x7f5554155070 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5553fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555415b8e0 0x7f555415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f55537fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55540cc820 0x7f55540c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5552ffd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5554155070 0x7f55540c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f55537fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55540cc820 0x7f55540c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:43240/0 (socket says 172.21.15.40:43240) 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5553fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555415b8e0 0x7f555415d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51340/0 (socket says 172.21.15.40:51340) 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f55537fe640 1 -- 172.21.15.40:0/730327003 learned_addr learned my addr 172.21.15.40:0/730327003 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5552ffd640 1 -- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55540cc820 msgr2=0x7f55540c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5552ffd640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55540cc820 0x7f55540c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5552ffd640 1 -- 172.21.15.40:0/730327003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555415b8e0 msgr2=0x7f555415d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5552ffd640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555415b8e0 0x7f555415d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5552ffd640 1 -- 172.21.15.40:0/730327003 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5554155c00 con 0x7f5554155070 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5552ffd640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5554155070 0x7f55540c5d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f555415bd50 tx=0x7f5548001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f553bfff640 1 -- 172.21.15.40:0/730327003 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548017070 con 0x7f5554155070 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f553bfff640 1 -- 172.21.15.40:0/730327003 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5548005ce0 con 0x7f5554155070 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55540c71b0 con 0x7f5554155070 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f553bfff640 1 -- 172.21.15.40:0/730327003 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55480072f0 con 0x7f5554155070 2024-04-25T11:38:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.691+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55540c4780 con 0x7f5554155070 2024-04-25T11:38:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.693+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5554068bb0 con 0x7f5554155070 2024-04-25T11:38:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.693+0000 7f553bfff640 1 -- 172.21.15.40:0/730327003 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f554801a070 con 0x7f5554155070 2024-04-25T11:38:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.693+0000 7f553bfff640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f554006c8f0 0x7f554006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.693+0000 7f553bfff640 1 -- 172.21.15.40:0/730327003 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f554808adb0 con 0x7f5554155070 2024-04-25T11:38:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.693+0000 7f55537fe640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f554006c8f0 0x7f554006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.694+0000 7f55537fe640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f554006c8f0 0x7f554006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55440097a0 tx=0x7f554402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.698+0000 7f553bfff640 1 -- 172.21.15.40:0/730327003 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f554808f070 con 0x7f5554155070 2024-04-25T11:38:51.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.880+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55540c7b90 con 0x7f5554155070 2024-04-25T11:38:51.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.880+0000 7f553bfff640 1 -- 172.21.15.40:0/730327003 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f5554155c00 con 0x7f5554155070 2024-04-25T11:38:51.868 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-25T11:38:51.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f554006c8f0 msgr2=0x7f554006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f554006c8f0 0x7f554006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55440097a0 tx=0x7f554402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5554155070 msgr2=0x7f55540c5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5554155070 0x7f55540c5d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f555415bd50 tx=0x7f5548001e70 comp rx=0 tx=0).stop 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 shutdown_connections 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f554006c8f0 0x7f554006edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f555415b8e0 0x7f555415d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5554155070 0x7f55540c5d90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 --2- 172.21.15.40:0/730327003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55540cc820 0x7f55540c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 >> 172.21.15.40:0/730327003 conn(0x7f555406c680 msgr2=0x7f555406df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 shutdown_connections 2024-04-25T11:38:51.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.882+0000 7f5559efd640 1 -- 172.21.15.40:0/730327003 wait complete. 2024-04-25T11:38:51.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:38:51.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.996+0000 7f608922a640 1 Processor -- start 2024-04-25T11:38:51.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.996+0000 7f608922a640 1 -- start start 2024-04-25T11:38:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f608922a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6084155230 0x7f60840cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f608922a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f608406ce40 0x7f60840cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f608922a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60840ccdb0 con 0x7f60840c56e0 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f608922a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f60840cb950 con 0x7f6084155230 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f608922a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f60840cc7f0 con 0x7f608406ce40 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f6082d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6084155230 0x7f60840cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f6082d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6084155230 0x7f60840cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51342/0 (socket says 172.21.15.40:51342) 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f607a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f60840c56e0 0x7f608415b800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53766/0 (socket says 172.21.15.40:53766) 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f6082d9d640 1 -- 172.21.15.40:0/2969539609 learned_addr learned my addr 172.21.15.40:0/2969539609 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f608359e640 1 --2- 172.21.15.40:0/2969539609 >> v2:172.21.15.62:3300/0 conn(0x7f608406ce40 0x7f60840cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f6082d9d640 1 -- 172.21.15.40:0/2969539609 >> v2:172.21.15.62:3300/0 conn(0x7f608406ce40 msgr2=0x7f60840cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f6082d9d640 1 --2- 172.21.15.40:0/2969539609 >> v2:172.21.15.62:3300/0 conn(0x7f608406ce40 0x7f60840cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f6082d9d640 1 -- 172.21.15.40:0/2969539609 >> v1:172.21.15.40:6789/0 conn(0x7f60840c56e0 legacy=0x7f608415b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f6082d9d640 1 -- 172.21.15.40:0/2969539609 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60840ccad0 con 0x7f6084155230 2024-04-25T11:38:51.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.997+0000 7f608259c640 1 -- 172.21.15.40:0/2969539609 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046416157 0 0) 0x7f606c001110 con 0x7f60840c56e0 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f6082d9d640 1 --2- 172.21.15.40:0/2969539609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6084155230 0x7f60840cb310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60840d16e0 tx=0x7f607402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ac5f97f6c01d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608259c640 1 -- 172.21.15.40:0/2969539609 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074037070 con 0x7f6084155230 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608259c640 1 -- 172.21.15.40:0/2969539609 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6074032440 con 0x7f6084155230 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608259c640 1 -- 172.21.15.40:0/2969539609 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074032810 con 0x7f6084155230 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608922a640 1 -- 172.21.15.40:0/2969539609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6084155230 msgr2=0x7f60840cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608922a640 1 --2- 172.21.15.40:0/2969539609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6084155230 0x7f60840cb310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60840d16e0 tx=0x7f607402a270 comp rx=0 tx=0).stop 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608922a640 1 -- 172.21.15.40:0/2969539609 shutdown_connections 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608922a640 1 --2- 172.21.15.40:0/2969539609 >> v2:172.21.15.62:3300/0 conn(0x7f608406ce40 0x7f60840cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608922a640 1 --2- 172.21.15.40:0/2969539609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6084155230 0x7f60840cb310 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608922a640 1 -- 172.21.15.40:0/2969539609 >> 172.21.15.40:0/2969539609 conn(0x7f6084067890 msgr2=0x7f6084067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608922a640 1 -- 172.21.15.40:0/2969539609 shutdown_connections 2024-04-25T11:38:51.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:51.998+0000 7f608922a640 1 -- 172.21.15.40:0/2969539609 wait complete. 2024-04-25T11:38:51.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.000+0000 7f608922a640 1 Processor -- start 2024-04-25T11:38:51.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.000+0000 7f608922a640 1 -- start start 2024-04-25T11:38:51.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f608922a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f608406ce40 0x7f608414e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f608922a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60840c56e0 0x7f60841f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f608922a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6084155230 0x7f608414af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f608922a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f608406d2d0 con 0x7f608406ce40 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f608922a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f608406c920 con 0x7f60840c56e0 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f608922a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f608406d450 con 0x7f6084155230 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f6082d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f608406ce40 0x7f608414e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f6082d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f608406ce40 0x7f608414e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51354/0 (socket says 172.21.15.40:51354) 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f6082d9d640 1 -- 172.21.15.40:0/3736880834 learned_addr learned my addr 172.21.15.40:0/3736880834 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f608359e640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6084155230 0x7f608414af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f607a59c640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60840c56e0 0x7f60841f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f6082d9d640 1 -- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6084155230 msgr2=0x7f608414af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f6082d9d640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6084155230 0x7f608414af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f6082d9d640 1 -- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60840c56e0 msgr2=0x7f60841f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f6082d9d640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60840c56e0 0x7f60841f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f6082d9d640 1 -- 172.21.15.40:0/3736880834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60840c79a0 con 0x7f608406ce40 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f6082d9d640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f608406ce40 0x7f608414e390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f607402a2b0 tx=0x7f60740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f607bfff640 1 -- 172.21.15.40:0/3736880834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074037040 con 0x7f608406ce40 2024-04-25T11:38:51.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f607bfff640 1 -- 172.21.15.40:0/3736880834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6074042070 con 0x7f608406ce40 2024-04-25T11:38:51.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60841f16c0 con 0x7f608406ce40 2024-04-25T11:38:51.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f607bfff640 1 -- 172.21.15.40:0/3736880834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074030880 con 0x7f608406ce40 2024-04-25T11:38:51.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.001+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60840c7f20 con 0x7f608406ce40 2024-04-25T11:38:51.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.002+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60841546a0 con 0x7f608406ce40 2024-04-25T11:38:51.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.003+0000 7f607bfff640 1 -- 172.21.15.40:0/3736880834 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6074030a20 con 0x7f608406ce40 2024-04-25T11:38:51.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.003+0000 7f607bfff640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f606406c8f0 0x7f606406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.003+0000 7f607bfff640 1 -- 172.21.15.40:0/3736880834 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f60740ac2b0 con 0x7f608406ce40 2024-04-25T11:38:51.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.003+0000 7f607a59c640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f606406c8f0 0x7f606406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.004+0000 7f607a59c640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f606406c8f0 0x7f606406edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f60840b6590 tx=0x7f606c007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.008+0000 7f607bfff640 1 -- 172.21.15.40:0/3736880834 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60841546a0 con 0x7f608406ce40 2024-04-25T11:38:52.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.192+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6084068bb0 con 0x7f608406ce40 2024-04-25T11:38:52.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.192+0000 7f607bfff640 1 -- 172.21.15.40:0/3736880834 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f60740ac0d0 con 0x7f608406ce40 2024-04-25T11:38:52.180 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-25T11:38:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f606406c8f0 msgr2=0x7f606406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f606406c8f0 0x7f606406edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f60840b6590 tx=0x7f606c007de0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f608406ce40 msgr2=0x7f608414e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f608406ce40 0x7f608414e390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f607402a2b0 tx=0x7f60740046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 shutdown_connections 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f606406c8f0 0x7f606406edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6084155230 0x7f608414af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60840c56e0 0x7f60841f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 --2- 172.21.15.40:0/3736880834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f608406ce40 0x7f608414e390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 >> 172.21.15.40:0/3736880834 conn(0x7f6084067890 msgr2=0x7f60841519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 shutdown_connections 2024-04-25T11:38:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.194+0000 7f608922a640 1 -- 172.21.15.40:0/3736880834 wait complete. 2024-04-25T11:38:52.192 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:52.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.305+0000 7f54f8423640 1 Processor -- start 2024-04-25T11:38:52.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.305+0000 7f54f8423640 1 -- start start 2024-04-25T11:38:52.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.306+0000 7f54f8423640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f0155010 0x7f54f00cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.306+0000 7f54f8423640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54f00c7af0 0x7f54f00cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.306+0000 7f54f8423640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f015afb0 con 0x7f54f00b6210 2024-04-25T11:38:52.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.306+0000 7f54f8423640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f54f00ccd30 con 0x7f54f00c7af0 2024-04-25T11:38:52.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.306+0000 7f54f8423640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f54f015b290 con 0x7f54f0155010 2024-04-25T11:38:52.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.306+0000 7f54f69c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54f00c7af0 0x7f54f00cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.306+0000 7f54f61bf640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f0155010 0x7f54f00cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.306+0000 7f54f59be640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f54f00b6210 0x7f54f00c89a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:45386/0 (socket says 172.21.15.40:45386) 2024-04-25T11:38:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.306+0000 7f54f59be640 1 -- 172.21.15.40:0/1923030252 learned_addr learned my addr 172.21.15.40:0/1923030252 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f51bd640 1 -- 172.21.15.40:0/1923030252 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595663380 0 0) 0x7f54f015afb0 con 0x7f54f00b6210 2024-04-25T11:38:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f51bd640 1 -- 172.21.15.40:0/1923030252 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54d8003520 con 0x7f54f00b6210 2024-04-25T11:38:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f69c0640 1 -- 172.21.15.40:0/1923030252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f0155010 msgr2=0x7f54f00cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f69c0640 1 --2- 172.21.15.40:0/1923030252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f0155010 0x7f54f00cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f69c0640 1 -- 172.21.15.40:0/1923030252 >> v1:172.21.15.62:6789/0 conn(0x7f54f00b6210 legacy=0x7f54f00c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f69c0640 1 -- 172.21.15.40:0/1923030252 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f015b570 con 0x7f54f00c7af0 2024-04-25T11:38:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f61bf640 1 --2- 172.21.15.40:0/1923030252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f0155010 0x7f54f00cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:52.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f69c0640 1 --2- 172.21.15.40:0/1923030252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00c7af0 0x7f54f00cb700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009d90 tx=0x7f54e002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e61d7f7f9f7dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f51bd640 1 -- 172.21.15.40:0/1923030252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0007740 con 0x7f54f00c7af0 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f51bd640 1 -- 172.21.15.40:0/1923030252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e00078e0 con 0x7f54f00c7af0 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.307+0000 7f54f51bd640 1 -- 172.21.15.40:0/1923030252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0007cb0 con 0x7f54f00c7af0 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.308+0000 7f54f8423640 1 -- 172.21.15.40:0/1923030252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00c7af0 msgr2=0x7f54f00cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.308+0000 7f54f8423640 1 --2- 172.21.15.40:0/1923030252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00c7af0 0x7f54f00cb700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009d90 tx=0x7f54e002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.308+0000 7f54f8423640 1 -- 172.21.15.40:0/1923030252 shutdown_connections 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.308+0000 7f54f8423640 1 --2- 172.21.15.40:0/1923030252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00c7af0 0x7f54f00cb700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.308+0000 7f54f8423640 1 --2- 172.21.15.40:0/1923030252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f0155010 0x7f54f00cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.308+0000 7f54f8423640 1 -- 172.21.15.40:0/1923030252 >> 172.21.15.40:0/1923030252 conn(0x7f54f00d0440 msgr2=0x7f54f00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.308+0000 7f54f8423640 1 -- 172.21.15.40:0/1923030252 shutdown_connections 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.308+0000 7f54f8423640 1 -- 172.21.15.40:0/1923030252 wait complete. 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.309+0000 7f54f8423640 1 Processor -- start 2024-04-25T11:38:52.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.309+0000 7f54f8423640 1 -- start start 2024-04-25T11:38:52.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f8423640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f00c7af0 0x7f54f0159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f8423640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f0155010 0x7f54f0155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f8423640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f0158480 0x7f54f0155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f8423640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f54f015ac80 con 0x7f54f0158480 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f8423640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f54f015bb30 con 0x7f54f0155010 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f8423640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f54f00c8040 con 0x7f54f00c7af0 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f69c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f0158480 0x7f54f0155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f69c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f0158480 0x7f54f0155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51366/0 (socket says 172.21.15.40:51366) 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f69c0640 1 -- 172.21.15.40:0/3381817382 learned_addr learned my addr 172.21.15.40:0/3381817382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f61bf640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f00c7af0 0x7f54f0159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f59be640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f0155010 0x7f54f0155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f69c0640 1 -- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f00c7af0 msgr2=0x7f54f0159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f69c0640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f00c7af0 0x7f54f0159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f69c0640 1 -- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f0155010 msgr2=0x7f54f0155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f69c0640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f0155010 0x7f54f0155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f69c0640 1 -- 172.21.15.40:0/3381817382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f01f07d0 con 0x7f54f0158480 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.310+0000 7f54f69c0640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f0158480 0x7f54f0155db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009eb0 tx=0x7f54e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.311+0000 7f54eeffd640 1 -- 172.21.15.40:0/3381817382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0040070 con 0x7f54f0158480 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.311+0000 7f54eeffd640 1 -- 172.21.15.40:0/3381817382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e0031bc0 con 0x7f54f0158480 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.311+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f01f0ab0 con 0x7f54f0158480 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.311+0000 7f54eeffd640 1 -- 172.21.15.40:0/3381817382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0009450 con 0x7f54f0158480 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.311+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01f1070 con 0x7f54f0158480 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.312+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f0068bb0 con 0x7f54f0158480 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.312+0000 7f54eeffd640 1 -- 172.21.15.40:0/3381817382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54e0039040 con 0x7f54f0158480 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.312+0000 7f54eeffd640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f54dc06c8f0 0x7f54dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.313+0000 7f54eeffd640 1 -- 172.21.15.40:0/3381817382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f54e00ad1c0 con 0x7f54f0158480 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.313+0000 7f54f61bf640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f54dc06c8f0 0x7f54dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.313+0000 7f54f61bf640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f54dc06c8f0 0x7f54dc06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009560 tx=0x7f54e8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.317+0000 7f54eeffd640 1 -- 172.21.15.40:0/3381817382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54e007b0b0 con 0x7f54f0158480 2024-04-25T11:38:52.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.540+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54f01555d0 con 0x7f54f0158480 2024-04-25T11:38:52.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.540+0000 7f54eeffd640 1 -- 172.21.15.40:0/3381817382 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6900 (secure 0 0 0) 0x7f54e007f3b0 con 0x7f54f0158480 2024-04-25T11:38:52.527 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:52.528 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1073487120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1073487120},{"type":"v1","addr":"172.21.15.40:6837","nonce":1073487120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:38:25.023156+0000","modified":"2024-04-25T11:38:48.036680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4300,"mds_1":4294,"mds_2":4279,"mds_3":4260,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"d","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.62:6838/3333589902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3333589902},{"type":"v1","addr":"172.21.15.62:6838","nonce":3333589902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4260":{"gid":4260,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4104143657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4104143657},{"type":"v1","addr":"172.21.15.40:6835","nonce":4104143657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6836/535060457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":535060457},{"type":"v1","addr":"172.21.15.40:6836","nonce":535060457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4294":{"gid":4294,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6835/3758619338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3758619338},{"type":"v1","addr":"172.21.15.62:6835","nonce":3758619338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/2121129816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6837","nonce":2121129816},{"type":"v1","addr":"172.21.15.62:6839","nonce":2121129816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4294,4279,4260]},"id":1}]} 2024-04-25T11:38:52.528 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-04-25T11:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f54dc06c8f0 msgr2=0x7f54dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f54dc06c8f0 0x7f54dc06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009560 tx=0x7f54e8009190 comp rx=0 tx=0).stop 2024-04-25T11:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f0158480 msgr2=0x7f54f0155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f0158480 0x7f54f0155db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009eb0 tx=0x7f54e0004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 shutdown_connections 2024-04-25T11:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f54dc06c8f0 0x7f54dc06edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f0158480 0x7f54f0155db0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f0155010 0x7f54f0155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 --2- 172.21.15.40:0/3381817382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f00c7af0 0x7f54f0159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 >> 172.21.15.40:0/3381817382 conn(0x7f54f00d0440 msgr2=0x7f54f0158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.542+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 shutdown_connections 2024-04-25T11:38:52.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.543+0000 7f54f8423640 1 -- 172.21.15.40:0/3381817382 wait complete. 2024-04-25T11:38:52.540 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.653+0000 7f0cf5072640 1 Processor -- start 2024-04-25T11:38:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.653+0000 7f0cf5072640 1 -- start start 2024-04-25T11:38:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cf5072640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0cf00cb440 0x7f0cf015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cf5072640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf00cbfe0 0x7f0cf00cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cf5072640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf006bc00 con 0x7f0cf0155000 2024-04-25T11:38:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cf5072640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf015bd10 con 0x7f0cf00cb440 2024-04-25T11:38:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cf5072640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf006bea0 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0ceed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0cf0155000 0x7f0cf006cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53768/0 (socket says 172.21.15.40:53768) 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0ceed9d640 1 -- 172.21.15.40:0/1181905284 learned_addr learned my addr 172.21.15.40:0/1181905284 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cee59c640 1 --2- 172.21.15.40:0/1181905284 >> v2:172.21.15.62:3300/0 conn(0x7f0cf00cb440 0x7f0cf015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cef59e640 1 --2- 172.21.15.40:0/1181905284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf00cbfe0 0x7f0cf00cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cedd9b640 1 -- 172.21.15.40:0/1181905284 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3024196507 0 0) 0x7f0cf006bc00 con 0x7f0cf0155000 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cedd9b640 1 -- 172.21.15.40:0/1181905284 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cdc003520 con 0x7f0cf0155000 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cee59c640 1 -- 172.21.15.40:0/1181905284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf00cbfe0 msgr2=0x7f0cf00cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cee59c640 1 --2- 172.21.15.40:0/1181905284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf00cbfe0 0x7f0cf00cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cee59c640 1 -- 172.21.15.40:0/1181905284 >> v1:172.21.15.40:6789/0 conn(0x7f0cf0155000 legacy=0x7f0cf006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.654+0000 7f0cee59c640 1 -- 172.21.15.40:0/1181905284 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf006c180 con 0x7f0cf00cb440 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.655+0000 7f0cee59c640 1 --2- 172.21.15.40:0/1181905284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cf00cb440 0x7f0cf015b7d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097a0 tx=0x7f0cd802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb7d4db350fe1de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.655+0000 7f0cedd9b640 1 -- 172.21.15.40:0/1181905284 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8037070 con 0x7f0cf00cb440 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.655+0000 7f0cedd9b640 1 -- 172.21.15.40:0/1181905284 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd8032440 con 0x7f0cf00cb440 2024-04-25T11:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.655+0000 7f0cedd9b640 1 -- 172.21.15.40:0/1181905284 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80327f0 con 0x7f0cf00cb440 2024-04-25T11:38:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.656+0000 7f0cf5072640 1 -- 172.21.15.40:0/1181905284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cf00cb440 msgr2=0x7f0cf015b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.656+0000 7f0cf5072640 1 --2- 172.21.15.40:0/1181905284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cf00cb440 0x7f0cf015b7d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097a0 tx=0x7f0cd802a210 comp rx=0 tx=0).stop 2024-04-25T11:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.656+0000 7f0cf5072640 1 -- 172.21.15.40:0/1181905284 shutdown_connections 2024-04-25T11:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.656+0000 7f0cf5072640 1 --2- 172.21.15.40:0/1181905284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf00cbfe0 0x7f0cf00cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.656+0000 7f0cf5072640 1 --2- 172.21.15.40:0/1181905284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cf00cb440 0x7f0cf015b7d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.656+0000 7f0cf5072640 1 -- 172.21.15.40:0/1181905284 >> 172.21.15.40:0/1181905284 conn(0x7f0cf00d0450 msgr2=0x7f0cf00d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.656+0000 7f0cf5072640 1 -- 172.21.15.40:0/1181905284 shutdown_connections 2024-04-25T11:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.656+0000 7f0cf5072640 1 -- 172.21.15.40:0/1181905284 wait complete. 2024-04-25T11:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.657+0000 7f0cf5072640 1 Processor -- start 2024-04-25T11:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.657+0000 7f0cf5072640 1 -- start start 2024-04-25T11:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cf5072640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cf00cb440 0x7f0cf015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cf5072640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf00cbfe0 0x7f0cf015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cf5072640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf0155000 0x7f0cf015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cf5072640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf00cb880 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cf5072640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf006c740 con 0x7f0cf00cb440 2024-04-25T11:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cf5072640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf00cbb60 con 0x7f0cf0155000 2024-04-25T11:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cee59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf00cbfe0 0x7f0cf015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cee59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf00cbfe0 0x7f0cf015a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51368/0 (socket says 172.21.15.40:51368) 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cee59c640 1 -- 172.21.15.40:0/735207647 learned_addr learned my addr 172.21.15.40:0/735207647 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0ceed9d640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cf00cb440 0x7f0cf015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cef59e640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf0155000 0x7f0cf015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cee59c640 1 -- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf0155000 msgr2=0x7f0cf015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cee59c640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf0155000 0x7f0cf015a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cee59c640 1 -- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cf00cb440 msgr2=0x7f0cf015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cee59c640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cf00cb440 0x7f0cf015b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.658+0000 7f0cee59c640 1 -- 172.21.15.40:0/735207647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf00c8160 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.659+0000 7f0cee59c640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf00cbfe0 0x7f0cf015a010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0cd802a250 tx=0x7f0cd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.659+0000 7f0ce77fe640 1 -- 172.21.15.40:0/735207647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8037040 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.659+0000 7f0ce77fe640 1 -- 172.21.15.40:0/735207647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd8042070 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.659+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf00c8400 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.659+0000 7f0ce77fe640 1 -- 172.21.15.40:0/735207647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd803b930 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.659+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf014b860 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.660+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf0068bb0 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.660+0000 7f0ce77fe640 1 -- 172.21.15.40:0/735207647 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cd803bad0 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.661+0000 7f0ce77fe640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cd006c8f0 0x7f0cd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.661+0000 7f0ce77fe640 1 -- 172.21.15.40:0/735207647 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0cd80ac230 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.661+0000 7f0ceed9d640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cd006c8f0 0x7f0cd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.661+0000 7f0ceed9d640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cd006c8f0 0x7f0cd006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005c00 tx=0x7f0ce0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.665+0000 7f0ce77fe640 1 -- 172.21.15.40:0/735207647 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0cd807a0a0 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.850+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cf00c9800 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.851+0000 7f0ce77fe640 1 -- 172.21.15.40:0/735207647 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f0cd807e3a0 con 0x7f0cf00cbfe0 2024-04-25T11:38:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:52.839 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:26.664101+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cd006c8f0 msgr2=0x7f0cd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cd006c8f0 0x7f0cd006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005c00 tx=0x7f0ce0009040 comp rx=0 tx=0).stop 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf00cbfe0 msgr2=0x7f0cf015a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf00cbfe0 0x7f0cf015a010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0cd802a250 tx=0x7f0cd80046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 shutdown_connections 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0cd006c8f0 0x7f0cd006edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cf0155000 0x7f0cf015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf00cbfe0 0x7f0cf015a010 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 --2- 172.21.15.40:0/735207647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cf00cb440 0x7f0cf015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 >> 172.21.15.40:0/735207647 conn(0x7f0cf00d0450 msgr2=0x7f0cf01509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 shutdown_connections 2024-04-25T11:38:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.853+0000 7f0cf5072640 1 -- 172.21.15.40:0/735207647 wait complete. 2024-04-25T11:38:52.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:52.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.964+0000 7fd31ed15640 1 Processor -- start 2024-04-25T11:38:52.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.964+0000 7fd31ed15640 1 -- start start 2024-04-25T11:38:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31ed15640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd318151f00 0x7fd3180c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31ed15640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31ed15640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3180c5580 con 0x7fd3181522e0 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31ed15640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd318158890 con 0x7fd318151f00 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31ed15640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd318158b70 con 0x7fd3180c8280 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31d2b2640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31cab1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd318151f00 0x7fd3180c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31cab1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd318151f00 0x7fd3180c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:55228/0 (socket says 172.21.15.40:55228) 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31d2b2640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:37642/0 (socket says 172.21.15.40:37642) 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31cab1640 1 -- 172.21.15.40:0/3029989309 learned_addr learned my addr 172.21.15.40:0/3029989309 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd317fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd3181522e0 0x7fd318157d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:39682/0 (socket says 172.21.15.40:39682) 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31d2b2640 1 -- 172.21.15.40:0/3029989309 >> v2:172.21.15.62:3300/0 conn(0x7fd318151f00 msgr2=0x7fd3180c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31d2b2640 1 --2- 172.21.15.40:0/3029989309 >> v2:172.21.15.62:3300/0 conn(0x7fd318151f00 0x7fd3180c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31d2b2640 1 -- 172.21.15.40:0/3029989309 >> v1:172.21.15.62:6789/0 conn(0x7fd3181522e0 legacy=0x7fd318157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31d2b2640 1 -- 172.21.15.40:0/3029989309 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3180c57f0 con 0x7fd3180c8280 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd3177fe640 1 -- 172.21.15.40:0/3029989309 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559374956 0 0) 0x7fd3180c5580 con 0x7fd3181522e0 2024-04-25T11:38:52.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.965+0000 7fd31cab1640 1 --2- 172.21.15.40:0/3029989309 >> v2:172.21.15.62:3300/0 conn(0x7fd318151f00 0x7fd3180c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.966+0000 7fd31d2b2640 1 --2- 172.21.15.40:0/3029989309 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180c6c00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd3000097a0 tx=0x7fd30002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a611c3341808c091 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.966+0000 7fd3177fe640 1 -- 172.21.15.40:0/3029989309 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300037070 con 0x7fd3180c8280 2024-04-25T11:38:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.966+0000 7fd3177fe640 1 -- 172.21.15.40:0/3029989309 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd300032440 con 0x7fd3180c8280 2024-04-25T11:38:52.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.966+0000 7fd3177fe640 1 -- 172.21.15.40:0/3029989309 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300032810 con 0x7fd3180c8280 2024-04-25T11:38:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.967+0000 7fd31ed15640 1 -- 172.21.15.40:0/3029989309 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 msgr2=0x7fd3180c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.967+0000 7fd31ed15640 1 --2- 172.21.15.40:0/3029989309 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180c6c00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd3000097a0 tx=0x7fd30002a210 comp rx=0 tx=0).stop 2024-04-25T11:38:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.967+0000 7fd31ed15640 1 -- 172.21.15.40:0/3029989309 shutdown_connections 2024-04-25T11:38:52.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.967+0000 7fd31ed15640 1 --2- 172.21.15.40:0/3029989309 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180c6c00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.967+0000 7fd31ed15640 1 --2- 172.21.15.40:0/3029989309 >> v2:172.21.15.62:3300/0 conn(0x7fd318151f00 0x7fd3180c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.967+0000 7fd31ed15640 1 -- 172.21.15.40:0/3029989309 >> 172.21.15.40:0/3029989309 conn(0x7fd31806b970 msgr2=0x7fd31806bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.967+0000 7fd31ed15640 1 -- 172.21.15.40:0/3029989309 shutdown_connections 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.967+0000 7fd31ed15640 1 -- 172.21.15.40:0/3029989309 wait complete. 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.968+0000 7fd31ed15640 1 Processor -- start 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.968+0000 7fd31ed15640 1 -- start start 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31ed15640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31ed15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd318151f00 0x7fd3180bb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31ed15640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3181522e0 0x7fd3180b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31ed15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd318158480 con 0x7fd318151f00 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31ed15640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd3180c5db0 con 0x7fd3181522e0 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31ed15640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd318158600 con 0x7fd3180c8280 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd317fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd318151f00 0x7fd3180bb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31cab1640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31cab1640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:37644/0 (socket says 172.21.15.40:37644) 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31cab1640 1 -- 172.21.15.40:0/111733152 learned_addr learned my addr 172.21.15.40:0/111733152 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31d2b2640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3181522e0 0x7fd3180b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31d2b2640 1 -- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 msgr2=0x7fd3180cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31d2b2640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31d2b2640 1 -- 172.21.15.40:0/111733152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd318151f00 msgr2=0x7fd3180bb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.969+0000 7fd31d2b2640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd318151f00 0x7fd3180bb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.970+0000 7fd31d2b2640 1 -- 172.21.15.40:0/111733152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3180ca280 con 0x7fd3181522e0 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.970+0000 7fd31cab1640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.970+0000 7fd31d2b2640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3181522e0 0x7fd3180b9bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd300031000 tx=0x7fd3000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.970+0000 7fd3157fa640 1 -- 172.21.15.40:0/111733152 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300041020 con 0x7fd3181522e0 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.970+0000 7fd3157fa640 1 -- 172.21.15.40:0/111733152 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30003d720 con 0x7fd3181522e0 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.970+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3180b8fb0 con 0x7fd3181522e0 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.970+0000 7fd3157fa640 1 -- 172.21.15.40:0/111733152 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300037040 con 0x7fd3181522e0 2024-04-25T11:38:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.970+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3180b9690 con 0x7fd3181522e0 2024-04-25T11:38:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.971+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd318073680 con 0x7fd3181522e0 2024-04-25T11:38:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.971+0000 7fd3157fa640 1 -- 172.21.15.40:0/111733152 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd300044020 con 0x7fd3181522e0 2024-04-25T11:38:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.972+0000 7fd3157fa640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd2f806c8f0 0x7fd2f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.972+0000 7fd3157fa640 1 -- 172.21.15.40:0/111733152 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd3000ac230 con 0x7fd3181522e0 2024-04-25T11:38:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.972+0000 7fd31cab1640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd2f806c8f0 0x7fd2f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.972+0000 7fd31cab1640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd2f806c8f0 0x7fd2f806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd3100072c0 tx=0x7fd31000d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:52.976+0000 7fd3157fa640 1 -- 172.21.15.40:0/111733152 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd318073680 con 0x7fd3181522e0 2024-04-25T11:38:53.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.209+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd318068bb0 con 0x7fd3181522e0 2024-04-25T11:38:53.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.210+0000 7fd3157fa640 1 -- 172.21.15.40:0/111733152 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6900 (secure 0 0 0) 0x7fd30004b030 con 0x7fd3181522e0 2024-04-25T11:38:53.197 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:53.197 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1073487120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1073487120},{"type":"v1","addr":"172.21.15.40:6837","nonce":1073487120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:38:25.023156+0000","modified":"2024-04-25T11:38:48.036680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4300,"mds_1":4294,"mds_2":4279,"mds_3":4260,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"d","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.62:6838/3333589902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3333589902},{"type":"v1","addr":"172.21.15.62:6838","nonce":3333589902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4260":{"gid":4260,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4104143657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4104143657},{"type":"v1","addr":"172.21.15.40:6835","nonce":4104143657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6836/535060457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":535060457},{"type":"v1","addr":"172.21.15.40:6836","nonce":535060457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4294":{"gid":4294,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6835/3758619338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3758619338},{"type":"v1","addr":"172.21.15.62:6835","nonce":3758619338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/2121129816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6837","nonce":2121129816},{"type":"v1","addr":"172.21.15.62:6839","nonce":2121129816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4294,4279,4260]},"id":1}]} 2024-04-25T11:38:53.197 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-04-25T11:38:53.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd2f806c8f0 msgr2=0x7fd2f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:53.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd2f806c8f0 0x7fd2f806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd3100072c0 tx=0x7fd31000d040 comp rx=0 tx=0).stop 2024-04-25T11:38:53.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3181522e0 msgr2=0x7fd3180b9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:53.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3181522e0 0x7fd3180b9bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd300031000 tx=0x7fd3000046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 shutdown_connections 2024-04-25T11:38:53.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd2f806c8f0 0x7fd2f806edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3181522e0 0x7fd3180b9bb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd318151f00 0x7fd3180bb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 --2- 172.21.15.40:0/111733152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3180c8280 0x7fd3180cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 >> 172.21.15.40:0/111733152 conn(0x7fd31806b970 msgr2=0x7fd3180c69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:53.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 shutdown_connections 2024-04-25T11:38:53.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:53.212+0000 7fd31ed15640 1 -- 172.21.15.40:0/111733152 wait complete. 2024-04-25T11:38:53.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:38:25.023156+0000', 'modified': '2024-04-25T11:38:48.036680+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4300, 'mds_1': 4294, 'mds_2': 4279, 'mds_3': 4260, 'mds_4': 4241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4241': {'gid': 4241, 'name': 'd', 'rank': 4, 'incarnation': 16, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.62:6838/3333589902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 3333589902}, {'type': 'v1', 'addr': '172.21.15.62:6838', 'nonce': 3333589902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4260': {'gid': 4260, 'name': 'e', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.40:6835/4104143657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 4104143657}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 4104143657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4279': {'gid': 4279, 'name': 'a', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.40:6836/535060457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 535060457}, {'type': 'v1', 'addr': '172.21.15.40:6836', 'nonce': 535060457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4294': {'gid': 4294, 'name': 'f', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.62:6835/3758619338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 3758619338}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 3758619338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4300': {'gid': 4300, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6839/2121129816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6837', 'nonce': 2121129816}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 2121129816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4300, 'qdb_cluster': [4300, 4294, 4279, 4260]} 2024-04-25T11:38:53.814 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4241:up:creating 2024-04-25T11:38:54.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:54.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.940+0000 7fb8e8220640 1 Processor -- start 2024-04-25T11:38:54.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.941+0000 7fb8e8220640 1 -- start start 2024-04-25T11:38:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.941+0000 7fb8e8220640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8e0155010 0x7fb8e00c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.941+0000 7fb8e8220640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb8e00c97f0 0x7fb8e015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.941+0000 7fb8e8220640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e01553f0 con 0x7fb8e006ca80 2024-04-25T11:38:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.941+0000 7fb8e8220640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e006eab0 con 0x7fb8e0155010 2024-04-25T11:38:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.941+0000 7fb8e8220640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e00c9300 con 0x7fb8e00c97f0 2024-04-25T11:38:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.942+0000 7fb8e5fbc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8e0155010 0x7fb8e00c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.942+0000 7fb8e5fbc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8e0155010 0x7fb8e00c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34650/0 (socket says 172.21.15.40:34650) 2024-04-25T11:38:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.942+0000 7fb8e5fbc640 1 -- 172.21.15.40:0/1089795771 learned_addr learned my addr 172.21.15.40:0/1089795771 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:54.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.942+0000 7fb8e57bb640 1 --2- 172.21.15.40:0/1089795771 >> v2:172.21.15.62:3300/0 conn(0x7fb8e00c97f0 0x7fb8e015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.942+0000 7fb8e5fbc640 1 -- 172.21.15.40:0/1089795771 >> v2:172.21.15.62:3300/0 conn(0x7fb8e00c97f0 msgr2=0x7fb8e015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:54.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.942+0000 7fb8e5fbc640 1 --2- 172.21.15.40:0/1089795771 >> v2:172.21.15.62:3300/0 conn(0x7fb8e00c97f0 0x7fb8e015eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.942+0000 7fb8e5fbc640 1 -- 172.21.15.40:0/1089795771 >> v1:172.21.15.62:6789/0 conn(0x7fb8e006ca80 legacy=0x7fb8e015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.942+0000 7fb8e5fbc640 1 -- 172.21.15.40:0/1089795771 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e00ce7c0 con 0x7fb8e0155010 2024-04-25T11:38:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e5fbc640 1 --2- 172.21.15.40:0/1089795771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e0155010 0x7fb8e00c8c50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006c0f0 tx=0x7fb8dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e560098e72c30a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e4fba640 1 -- 172.21.15.40:0/1089795771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc037070 con 0x7fb8e0155010 2024-04-25T11:38:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e4fba640 1 -- 172.21.15.40:0/1089795771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8dc032440 con 0x7fb8e0155010 2024-04-25T11:38:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e4fba640 1 -- 172.21.15.40:0/1089795771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc032810 con 0x7fb8e0155010 2024-04-25T11:38:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e8220640 1 -- 172.21.15.40:0/1089795771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e0155010 msgr2=0x7fb8e00c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e8220640 1 --2- 172.21.15.40:0/1089795771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e0155010 0x7fb8e00c8c50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006c0f0 tx=0x7fb8dc02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e8220640 1 -- 172.21.15.40:0/1089795771 shutdown_connections 2024-04-25T11:38:54.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e8220640 1 --2- 172.21.15.40:0/1089795771 >> v2:172.21.15.62:3300/0 conn(0x7fb8e00c97f0 0x7fb8e015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e8220640 1 --2- 172.21.15.40:0/1089795771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e0155010 0x7fb8e00c8c50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.943+0000 7fb8e8220640 1 -- 172.21.15.40:0/1089795771 >> 172.21.15.40:0/1089795771 conn(0x7fb8e006b970 msgr2=0x7fb8e006f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.944+0000 7fb8e8220640 1 -- 172.21.15.40:0/1089795771 shutdown_connections 2024-04-25T11:38:54.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.944+0000 7fb8e8220640 1 -- 172.21.15.40:0/1089795771 wait complete. 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.945+0000 7fb8e8220640 1 Processor -- start 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.945+0000 7fb8e8220640 1 -- start start 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.945+0000 7fb8e8220640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006ca80 0x7fb8e015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.945+0000 7fb8e8220640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e00c97f0 0x7fb8e01e5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.945+0000 7fb8e8220640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e0155010 0x7fb8e01e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.945+0000 7fb8e8220640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e015e310 con 0x7fb8e00c97f0 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.945+0000 7fb8e8220640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00ced80 con 0x7fb8e0155010 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.945+0000 7fb8e8220640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e015e5f0 con 0x7fb8e006ca80 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e57bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e00c97f0 0x7fb8e01e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e57bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e00c97f0 0x7fb8e01e5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34664/0 (socket says 172.21.15.40:34664) 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e5fbc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006ca80 0x7fb8e015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e57bb640 1 -- 172.21.15.40:0/1970604716 learned_addr learned my addr 172.21.15.40:0/1970604716 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:54.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e67bd640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e0155010 0x7fb8e01e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e57bb640 1 -- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006ca80 msgr2=0x7fb8e015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e57bb640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006ca80 0x7fb8e015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e57bb640 1 -- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e0155010 msgr2=0x7fb8e01e55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e57bb640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e0155010 0x7fb8e01e55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e57bb640 1 -- 172.21.15.40:0/1970604716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e01534e0 con 0x7fb8e00c97f0 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e67bd640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e0155010 0x7fb8e01e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e57bb640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e00c97f0 0x7fb8e01e5070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000e8a0 tx=0x7fb8d000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8d67fc640 1 -- 172.21.15.40:0/1970604716 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0017070 con 0x7fb8e00c97f0 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8d67fc640 1 -- 172.21.15.40:0/1970604716 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d000c940 con 0x7fb8e00c97f0 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e00cabb0 con 0x7fb8e00c97f0 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8d67fc640 1 -- 172.21.15.40:0/1970604716 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000cd30 con 0x7fb8e00c97f0 2024-04-25T11:38:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.946+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e00cb130 con 0x7fb8e00c97f0 2024-04-25T11:38:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.947+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e004fdd0 con 0x7fb8e00c97f0 2024-04-25T11:38:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.948+0000 7fb8d67fc640 1 -- 172.21.15.40:0/1970604716 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8d001a070 con 0x7fb8e00c97f0 2024-04-25T11:38:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.948+0000 7fb8d67fc640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb8c406c920 0x7fb8c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.948+0000 7fb8d67fc640 1 -- 172.21.15.40:0/1970604716 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8d008b8f0 con 0x7fb8e00c97f0 2024-04-25T11:38:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.948+0000 7fb8e5fbc640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb8c406c920 0x7fb8c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.949+0000 7fb8e5fbc640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb8c406c920 0x7fb8c406ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc034000 tx=0x7fb8dc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:54.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:54.953+0000 7fb8d67fc640 1 -- 172.21.15.40:0/1970604716 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8e004fdd0 con 0x7fb8e00c97f0 2024-04-25T11:38:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.177+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8e0068bb0 con 0x7fb8e00c97f0 2024-04-25T11:38:55.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.177+0000 7fb8d67fc640 1 -- 172.21.15.40:0/1970604716 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6904 (secure 0 0 0) 0x7fb8d00596d0 con 0x7fb8e00c97f0 2024-04-25T11:38:55.165 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:38:55.165 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1073487120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1073487120},{"type":"v1","addr":"172.21.15.40:6837","nonce":1073487120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:38:25.023156+0000","modified":"2024-04-25T11:38:54.056041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4300,"mds_1":4294,"mds_2":4279,"mds_3":4260,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"d","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6838/3333589902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3333589902},{"type":"v1","addr":"172.21.15.62:6838","nonce":3333589902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4260":{"gid":4260,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4104143657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4104143657},{"type":"v1","addr":"172.21.15.40:6835","nonce":4104143657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6836/535060457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":535060457},{"type":"v1","addr":"172.21.15.40:6836","nonce":535060457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4294":{"gid":4294,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6835/3758619338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3758619338},{"type":"v1","addr":"172.21.15.62:6835","nonce":3758619338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/2121129816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6837","nonce":2121129816},{"type":"v1","addr":"172.21.15.62:6839","nonce":2121129816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4241,4260,4279,4294,4300]},"id":1}]} 2024-04-25T11:38:55.165 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb8c406c920 msgr2=0x7fb8c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb8c406c920 0x7fb8c406ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc034000 tx=0x7fb8dc02e000 comp rx=0 tx=0).stop 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e00c97f0 msgr2=0x7fb8e01e5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e00c97f0 0x7fb8e01e5070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000e8a0 tx=0x7fb8d000ed60 comp rx=0 tx=0).stop 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 shutdown_connections 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fb8c406c920 0x7fb8c406ede0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e0155010 0x7fb8e01e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e00c97f0 0x7fb8e01e5070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 --2- 172.21.15.40:0/1970604716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006ca80 0x7fb8e015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 >> 172.21.15.40:0/1970604716 conn(0x7fb8e006b970 msgr2=0x7fb8e00c8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:55.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 shutdown_connections 2024-04-25T11:38:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.179+0000 7fb8e8220640 1 -- 172.21.15.40:0/1970604716 wait complete. 2024-04-25T11:38:55.255 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-04-25T11:38:55.266+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T11:38:55.781 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:38:25.023156+0000', 'modified': '2024-04-25T11:38:54.056041+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4300, 'mds_1': 4294, 'mds_2': 4279, 'mds_3': 4260, 'mds_4': 4241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4241': {'gid': 4241, 'name': 'd', 'rank': 4, 'incarnation': 16, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.62:6838/3333589902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 3333589902}, {'type': 'v1', 'addr': '172.21.15.62:6838', 'nonce': 3333589902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4260': {'gid': 4260, 'name': 'e', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.40:6835/4104143657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 4104143657}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 4104143657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4279': {'gid': 4279, 'name': 'a', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.40:6836/535060457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 535060457}, {'type': 'v1', 'addr': '172.21.15.40:6836', 'nonce': 535060457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4294': {'gid': 4294, 'name': 'f', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.62:6835/3758619338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 3758619338}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 3758619338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4300': {'gid': 4300, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6839/2121129816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6837', 'nonce': 2121129816}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 2121129816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4300, 'qdb_cluster': [4241, 4260, 4279, 4294, 4300]} 2024-04-25T11:38:55.781 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T11:38:55.782 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T11:38:55.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.909+0000 7f8db4a4e640 1 Processor -- start 2024-04-25T11:38:55.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.909+0000 7f8db4a4e640 1 -- start start 2024-04-25T11:38:55.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.909+0000 7f8db4a4e640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8db00cb440 0x7f8db015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.909+0000 7f8db4a4e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8db00cbfe0 0x7f8db00cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.909+0000 7f8db4a4e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8db006bc00 con 0x7f8db0155000 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.909+0000 7f8db4a4e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8db015bd10 con 0x7f8db00cbfe0 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.909+0000 7f8db4a4e640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8db006bea0 con 0x7f8db00cb440 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8daed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8db00cbfe0 0x7f8db00cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8da7fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8db00cb440 0x7f8db015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8dae59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8db0155000 0x7f8db006cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37626/0 (socket says 172.21.15.40:37626) 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8daed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8db00cbfe0 0x7f8db00cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34674/0 (socket says 172.21.15.40:34674) 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8dae59c640 1 -- 172.21.15.40:0/2143669848 learned_addr learned my addr 172.21.15.40:0/2143669848 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8da7fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8db00cb440 0x7f8db015b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:55268/0 (socket says 172.21.15.40:55268) 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8da7fff640 1 -- 172.21.15.40:0/2143669848 >> v2:172.21.15.40:3300/0 conn(0x7f8db00cbfe0 msgr2=0x7f8db00cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8da7fff640 1 --2- 172.21.15.40:0/2143669848 >> v2:172.21.15.40:3300/0 conn(0x7f8db00cbfe0 0x7f8db00cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8da7fff640 1 -- 172.21.15.40:0/2143669848 >> v1:172.21.15.40:6789/0 conn(0x7f8db0155000 legacy=0x7f8db006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8da7fff640 1 -- 172.21.15.40:0/2143669848 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db006c180 con 0x7f8db00cb440 2024-04-25T11:38:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.910+0000 7f8daed9d640 1 --2- 172.21.15.40:0/2143669848 >> v2:172.21.15.40:3300/0 conn(0x7f8db00cbfe0 0x7f8db00cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8da7fff640 1 --2- 172.21.15.40:0/2143669848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db00cb440 0x7f8db015b7d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8db0069cb0 tx=0x7f8d9802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4208e771a53cf89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8dadd9b640 1 -- 172.21.15.40:0/2143669848 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98037070 con 0x7f8db00cb440 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8dadd9b640 1 -- 172.21.15.40:0/2143669848 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d98032440 con 0x7f8db00cb440 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8dadd9b640 1 -- 172.21.15.40:0/2143669848 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98032810 con 0x7f8db00cb440 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8db4a4e640 1 -- 172.21.15.40:0/2143669848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db00cb440 msgr2=0x7f8db015b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8db4a4e640 1 --2- 172.21.15.40:0/2143669848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db00cb440 0x7f8db015b7d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8db0069cb0 tx=0x7f8d9802a250 comp rx=0 tx=0).stop 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8db4a4e640 1 -- 172.21.15.40:0/2143669848 shutdown_connections 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8db4a4e640 1 --2- 172.21.15.40:0/2143669848 >> v2:172.21.15.40:3300/0 conn(0x7f8db00cbfe0 0x7f8db00cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8db4a4e640 1 --2- 172.21.15.40:0/2143669848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db00cb440 0x7f8db015b7d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8db4a4e640 1 -- 172.21.15.40:0/2143669848 >> 172.21.15.40:0/2143669848 conn(0x7f8db00d0450 msgr2=0x7f8db00d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8db4a4e640 1 -- 172.21.15.40:0/2143669848 shutdown_connections 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.911+0000 7f8db4a4e640 1 -- 172.21.15.40:0/2143669848 wait complete. 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.913+0000 7f8db4a4e640 1 Processor -- start 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.913+0000 7f8db4a4e640 1 -- start start 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.913+0000 7f8db4a4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db00cb440 0x7f8db0149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.913+0000 7f8db4a4e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db00cbfe0 0x7f8db014a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.913+0000 7f8db4a4e640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8db0155000 0x7f8db014ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.913+0000 7f8db4a4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8db00cb880 con 0x7f8db00cb440 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.913+0000 7f8db4a4e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8db006c740 con 0x7f8db00cbfe0 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.913+0000 7f8db4a4e640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8db00cbb60 con 0x7f8db0155000 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db00cb440 0x7f8db0149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db00cb440 0x7f8db0149e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34680/0 (socket says 172.21.15.40:34680) 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8dae59c640 1 -- 172.21.15.40:0/3253550120 learned_addr learned my addr 172.21.15.40:0/3253550120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8da7fff640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db00cbfe0 0x7f8db014a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8daed9d640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8db0155000 0x7f8db014ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8dae59c640 1 -- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8db0155000 msgr2=0x7f8db014ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8dae59c640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8db0155000 0x7f8db014ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8dae59c640 1 -- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db00cbfe0 msgr2=0x7f8db014a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8dae59c640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db00cbfe0 0x7f8db014a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8dae59c640 1 -- 172.21.15.40:0/3253550120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db0148b40 con 0x7f8db00cb440 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8dae59c640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db00cb440 0x7f8db0149e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8da000c3c0 tx=0x7f8da000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3253550120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0017070 con 0x7f8db00cb440 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3253550120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da0005ce0 con 0x7f8db00cb440 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3253550120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da00072f0 con 0x7f8db00cb440 2024-04-25T11:38:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8db4a4e640 1 -- 172.21.15.40:0/3253550120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db0148de0 con 0x7f8db00cb440 2024-04-25T11:38:55.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.914+0000 7f8db4a4e640 1 -- 172.21.15.40:0/3253550120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db01493a0 con 0x7f8db00cb440 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.916+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3253550120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8da001a070 con 0x7f8db00cb440 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.916+0000 7f8da6ffd640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8d9006c920 0x7f8d9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.916+0000 7f8da6ffd640 1 -- 172.21.15.40:0/3253550120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8da008ac90 con 0x7f8db00cb440 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.916+0000 7f8da7fff640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8d9006c920 0x7f8d9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.917+0000 7f8da7fff640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8d9006c920 0x7f8d9006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d98034000 tx=0x7f8d9802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.917+0000 7f8db7764740 1 Processor -- start 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8db7764740 1 -- start start 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8db7764740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2beba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8db7764740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba6fc0 0x55d0e2baaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8db7764740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2bf58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8db7764740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2b10c50 con 0x55d0e2ba6fc0 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8db7764740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2aad830 con 0x55d0e2ba7690 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8db7764740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d0e2b0cc10 con 0x55d0e2ba7a70 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba6fc0 0x55d0e2baaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2beba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba6fc0 0x55d0e2baaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34692/0 (socket says 172.21.15.40:34692) 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8da7fff640 1 -- 172.21.15.40:0/1639836527 learned_addr learned my addr 172.21.15.40:0/1639836527 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.918+0000 7f8daed9d640 1 --2- 172.21.15.40:0/1639836527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2bf58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8da7fff640 1 -- 172.21.15.40:0/1639836527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 msgr2=0x55d0e2bf58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8da7fff640 1 --2- 172.21.15.40:0/1639836527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2bf58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8da7fff640 1 -- 172.21.15.40:0/1639836527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 msgr2=0x55d0e2beba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8da7fff640 1 --2- 172.21.15.40:0/1639836527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2beba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8da7fff640 1 -- 172.21.15.40:0/1639836527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e28232d0 con 0x55d0e2ba6fc0 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8dae59c640 1 --2- 172.21.15.40:0/1639836527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2beba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8da7fff640 1 --2- 172.21.15.40:0/1639836527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba6fc0 0x55d0e2baaf40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d9803e040 tx=0x7f8d9802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd2efd745b70733 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8d8ffff640 1 -- 172.21.15.40:0/1639836527 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98037040 con 0x55d0e2ba6fc0 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8d8ffff640 1 -- 172.21.15.40:0/1639836527 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d9804e070 con 0x55d0e2ba6fc0 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.919+0000 7f8d8ffff640 1 -- 172.21.15.40:0/1639836527 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98032de0 con 0x55d0e2ba6fc0 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 -- 172.21.15.40:0/1639836527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba6fc0 msgr2=0x55d0e2baaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 --2- 172.21.15.40:0/1639836527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba6fc0 0x55d0e2baaf40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d9803e040 tx=0x7f8d9802c000 comp rx=0 tx=0).stop 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 -- 172.21.15.40:0/1639836527 shutdown_connections 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 --2- 172.21.15.40:0/1639836527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2bf58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 --2- 172.21.15.40:0/1639836527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba6fc0 0x55d0e2baaf40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 --2- 172.21.15.40:0/1639836527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2beba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 -- 172.21.15.40:0/1639836527 >> 172.21.15.40:0/1639836527 conn(0x55d0e2ba8c80 msgr2=0x55d0e2bab580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 -- 172.21.15.40:0/1639836527 shutdown_connections 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 -- 172.21.15.40:0/1639836527 wait complete. 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 -- start start 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.920+0000 7f8db7764740 1 Processor -- start 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8db7764740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2bcf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8db7764740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2bcd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8db7764740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bcc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8db7764740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2bec700 con 0x55d0e2bcebe0 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8db7764740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2aab470 con 0x55d0e2ba7690 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8db7764740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d0e2baa0e0 con 0x55d0e2ba7a70 2024-04-25T11:38:55.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8db7764740 10 client.0 _ll_register_callbacks cb 0x55d0e2ad0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8daed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bcc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2bcf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8daed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bcc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34706/0 (socket says 172.21.15.40:34706) 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2bcf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:55290/0 (socket says 172.21.15.40:55290) 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8daed9d640 1 -- 172.21.15.40:0/89726268 learned_addr learned my addr 172.21.15.40:0/89726268 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.921+0000 7f8da7fff640 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2bcd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8daed9d640 1 -- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 msgr2=0x55d0e2bcd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8daed9d640 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2bcd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8daed9d640 1 -- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 msgr2=0x55d0e2bcf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8daed9d640 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2bcf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8daed9d640 1 -- 172.21.15.40:0/89726268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e28ed6b0 con 0x55d0e2bcebe0 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8dae59c640 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2bcf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8daed9d640 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bcc5d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8da8014d90 tx=0x7f8da8012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1532810cf9efe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8d8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/89726268 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da801e620 con 0x55d0e2bcebe0 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/89726268 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8da801e7c0 con 0x55d0e2bcebe0 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8db7764740 1 -- 172.21.15.40:0/89726268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d9804e070 con 0x55d0e2bcebe0 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/89726268 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8da801e9c0 con 0x55d0e2bcebe0 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8db7764740 10 client.4704 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8db7764740 1 -- 172.21.15.40:0/89726268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d0e2bcff10 con 0x55d0e2bcebe0 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.922+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/89726268 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d9804e070 con 0x55d0e2bcebe0 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/89726268 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f8da802b080 con 0x55d0e2bcebe0 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8db7764740 10 client.4704 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8db7764740 10 client.4704 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8db7764740 20 client.4704 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:55.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8db7764740 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] conn(0x55d0e2c0a360 0x55d0e2c0c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8db7764740 4 client.4704 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:38:55.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8db7764740 1 -- 172.21.15.40:0/89726268 --> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0e2b1e670 con 0x55d0e2c0a360 2024-04-25T11:38:55.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8dae59c640 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] conn(0x55d0e2c0a360 0x55d0e2c0c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8dae59c640 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] conn(0x55d0e2c0a360 0x55d0e2c0c750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=28b310ab654b411f server_cookie=57ce2f7fe3ecaf8 in_seq=0 out_seq=0 2024-04-25T11:38:55.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:55.923+0000 7f8d8d7fa640 10 client.4704 ms_handle_connect on v2:172.21.15.62:6836/3333589902 2024-04-25T11:38:57.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:38:57.711 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:38:59.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.061+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/89726268 <== mds.4 v2:172.21.15.62:6836/3333589902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0e2b1e670 con 0x55d0e2c0a360 2024-04-25T11:38:59.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.061+0000 7f8d8d7fa640 10 client.4704 handle_command_reply: tid=0 2024-04-25T11:38:59.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.061+0000 7f8db7764740 1 client.4704 shutdown 2024-04-25T11:38:59.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.061+0000 7f8db7764740 1 -- 172.21.15.40:0/89726268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2bcebe0 msgr2=0x55d0e2bcc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.061+0000 7f8db7764740 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bcc5d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8da8014d90 tx=0x7f8da8012670 comp rx=0 tx=0).stop 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.061+0000 7f8db7764740 1 -- 172.21.15.40:0/89726268 shutdown_connections 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.061+0000 7f8db7764740 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] conn(0x55d0e2c0a360 0x55d0e2c0c750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.061+0000 7f8db7764740 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bcc5d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.062+0000 7f8db7764740 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2bcd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.062+0000 7f8db7764740 1 --2- 172.21.15.40:0/89726268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2bcf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.062+0000 7f8d8d7fa640 0 client.4704 ms_handle_reset on v2:172.21.15.62:6836/3333589902 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.062+0000 7f8db7764740 1 -- 172.21.15.40:0/89726268 >> 172.21.15.40:0/89726268 conn(0x55d0e2ba6fc0 msgr2=0x55d0e2ba61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.062+0000 7f8db7764740 1 -- 172.21.15.40:0/89726268 shutdown_connections 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.062+0000 7f8db7764740 1 -- 172.21.15.40:0/89726268 wait complete. 2024-04-25T11:38:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.062+0000 7f8db7764740 20 client.4704 trim_cache size 0 max 16384 2024-04-25T11:38:59.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.082+0000 7f8db7764740 1 Processor -- start 2024-04-25T11:38:59.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.082+0000 7f8db7764740 1 -- start start 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8db7764740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7690 0x55d0e2bec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8db7764740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2becc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8db7764740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bc6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8db7764740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8db0149b30 con 0x55d0e2ba7690 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8db7764740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2bfde20 con 0x55d0e2bcebe0 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8db7764740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d0e2bfd140 con 0x55d0e2ba7a70 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7690 0x55d0e2bec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7690 0x55d0e2bec700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34708/0 (socket says 172.21.15.40:34708) 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8dae59c640 1 -- 172.21.15.40:0/2427273405 learned_addr learned my addr 172.21.15.40:0/2427273405 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8da7fff640 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2becc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8daed9d640 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bc6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8dae59c640 1 -- 172.21.15.40:0/2427273405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 msgr2=0x55d0e2becc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8dae59c640 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2becc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8dae59c640 1 -- 172.21.15.40:0/2427273405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2bcebe0 msgr2=0x55d0e2bc6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8dae59c640 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bc6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8dae59c640 1 -- 172.21.15.40:0/2427273405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e2bebf50 con 0x55d0e2ba7690 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8daed9d640 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bc6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.083+0000 7f8dae59c640 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7690 0x55d0e2bec700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8da000ccc0 tx=0x7f8da005f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130c455e490bbe02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.084+0000 7f8d8f7fe640 1 -- 172.21.15.40:0/2427273405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0082bb0 con 0x55d0e2ba7690 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.084+0000 7f8d8f7fe640 1 -- 172.21.15.40:0/2427273405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da0082d50 con 0x55d0e2ba7690 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.084+0000 7f8d8f7fe640 1 -- 172.21.15.40:0/2427273405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da005f2b0 con 0x55d0e2ba7690 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.084+0000 7f8db7764740 1 -- 172.21.15.40:0/2427273405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7690 msgr2=0x55d0e2bec700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.084+0000 7f8db7764740 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7690 0x55d0e2bec700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8da000ccc0 tx=0x7f8da005f040 comp rx=0 tx=0).stop 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.085+0000 7f8db7764740 1 -- 172.21.15.40:0/2427273405 shutdown_connections 2024-04-25T11:38:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.085+0000 7f8db7764740 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2bcebe0 0x55d0e2bc6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.085+0000 7f8db7764740 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2ba7a70 0x55d0e2becc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.085+0000 7f8db7764740 1 --2- 172.21.15.40:0/2427273405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7690 0x55d0e2bec700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.085+0000 7f8db7764740 1 -- 172.21.15.40:0/2427273405 >> 172.21.15.40:0/2427273405 conn(0x55d0e2ba6fc0 msgr2=0x55d0e2bcf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.085+0000 7f8db7764740 1 -- 172.21.15.40:0/2427273405 shutdown_connections 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.085+0000 7f8db7764740 1 -- 172.21.15.40:0/2427273405 wait complete. 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.085+0000 7f8db7764740 1 -- start start 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.085+0000 7f8db7764740 1 Processor -- start 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.086+0000 7f8db7764740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2c6b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.086+0000 7f8db7764740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7a70 0x55d0e2bac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.086+0000 7f8db7764740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2bcebe0 0x55d0e2c64a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.086+0000 7f8db7764740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2bd53a0 con 0x55d0e2ba7a70 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.086+0000 7f8db7764740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2bccd80 con 0x55d0e2ba7690 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.086+0000 7f8db7764740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d0e2bd5a10 con 0x55d0e2bcebe0 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.086+0000 7f8db7764740 10 client.0 _ll_register_callbacks cb 0x7f8da80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7a70 0x55d0e2bac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7a70 0x55d0e2bac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34710/0 (socket says 172.21.15.40:34710) 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8dae59c640 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2c6b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8da7fff640 1 -- 172.21.15.40:0/351632149 learned_addr learned my addr 172.21.15.40:0/351632149 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8daed9d640 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2bcebe0 0x55d0e2c64a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8dae59c640 1 -- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2bcebe0 msgr2=0x55d0e2c64a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8dae59c640 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2bcebe0 0x55d0e2c64a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8dae59c640 1 -- 172.21.15.40:0/351632149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7a70 msgr2=0x55d0e2bac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8dae59c640 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7a70 0x55d0e2bac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8dae59c640 1 -- 172.21.15.40:0/351632149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e2bed280 con 0x55d0e2ba7690 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8da7fff640 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7a70 0x55d0e2bac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8dae59c640 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2c6b300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8da0058800 tx=0x7f8da0000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57702a6b2c22a5dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8d8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/351632149 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0082bb0 con 0x55d0e2ba7690 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/351632149 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8da0089220 con 0x55d0e2ba7690 2024-04-25T11:38:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8db7764740 1 -- 172.21.15.40:0/351632149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da0082d50 con 0x55d0e2ba7690 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.087+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/351632149 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8da0007490 con 0x55d0e2ba7690 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8db7764740 10 client.4672 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8db7764740 1 -- 172.21.15.40:0/351632149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d0e2c6bb30 con 0x55d0e2ba7690 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/351632149 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da0082d50 con 0x55d0e2ba7690 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/351632149 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f8da0057f50 con 0x55d0e2ba7690 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8db7764740 10 client.4672 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8db7764740 10 client.4672 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8db7764740 20 client.4672 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8db7764740 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] conn(0x55d0e2c680b0 0x55d0e2c68490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8db7764740 4 client.4672 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8db7764740 1 -- 172.21.15.40:0/351632149 --> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] -- command(tid 0: {"prefix": "status"}) -- 0x55d0e2b184e0 con 0x55d0e2c680b0 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8da7fff640 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] conn(0x55d0e2c680b0 0x55d0e2c68490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8da7fff640 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] conn(0x55d0e2c680b0 0x55d0e2c68490 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ea6a6d0c38aa383 server_cookie=627dff1af1f9846 in_seq=0 out_seq=0 2024-04-25T11:38:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.088+0000 7f8d8d7fa640 10 client.4672 ms_handle_connect on v2:172.21.15.62:6836/3333589902 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8d8d7fa640 1 -- 172.21.15.40:0/351632149 <== mds.4 v2:172.21.15.62:6836/3333589902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55d0e2b184e0 con 0x55d0e2c680b0 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8d8d7fa640 10 client.4672 handle_command_reply: tid=0 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 client.4672 shutdown 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 -- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 msgr2=0x55d0e2c6b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2c6b300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8da0058800 tx=0x7f8da0000ed0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 -- 172.21.15.40:0/351632149 shutdown_connections 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:6836/3333589902,v1:172.21.15.62:6838/3333589902] conn(0x55d0e2c680b0 0x55d0e2c68490 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d0e2bcebe0 0x55d0e2c64a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d0e2ba7a70 0x55d0e2bac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8d8d7fa640 0 client.4672 ms_handle_reset on v2:172.21.15.62:6836/3333589902 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 --2- 172.21.15.40:0/351632149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d0e2ba7690 0x55d0e2c6b300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 -- 172.21.15.40:0/351632149 >> 172.21.15.40:0/351632149 conn(0x55d0e2ba6fc0 msgr2=0x55d0e2bd0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 -- 172.21.15.40:0/351632149 shutdown_connections 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 1 -- 172.21.15.40:0/351632149 wait complete. 2024-04-25T11:38:59.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.089+0000 7f8db7764740 20 client.4672 trim_cache size 0 max 16384 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "6d14dde7-764d-42d0-be37-c790a704858d", 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 4, 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4241, 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 11.048570427, 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 17, 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 20, 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T11:38:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 39.816578235999998 2024-04-25T11:38:59.085 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 -- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8d9006c920 msgr2=0x7f8d9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8d9006c920 0x7f8d9006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d98034000 tx=0x7f8d9802e000 comp rx=0 tx=0).stop 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 -- 172.21.15.40:0/3253550120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db00cb440 msgr2=0x7f8db0149e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db00cb440 0x7f8db0149e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8da000c3c0 tx=0x7f8da000c7c0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 -- 172.21.15.40:0/3253550120 shutdown_connections 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f8d9006c920 0x7f8d9006ede0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8db0155000 0x7f8db014ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db00cbfe0 0x7f8db014a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 --2- 172.21.15.40:0/3253550120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db00cb440 0x7f8db0149e80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 -- 172.21.15.40:0/3253550120 >> 172.21.15.40:0/3253550120 conn(0x7f8db00d0450 msgr2=0x7f8db01508e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 -- 172.21.15.40:0/3253550120 shutdown_connections 2024-04-25T11:38:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.090+0000 7f8db4a4e640 1 -- 172.21.15.40:0/3253550120 wait complete. 2024-04-25T11:38:59.091 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-25T11:38:59.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.211+0000 7fc9c4908640 1 Processor -- start 2024-04-25T11:38:59.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.211+0000 7fc9c4908640 1 -- start start 2024-04-25T11:38:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.212+0000 7fc9c4908640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c0154f50 0x7fc9c00c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.212+0000 7fc9c4908640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9c00b6210 0x7fc9c00cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.212+0000 7fc9c4908640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c015f060 con 0x7fc9c00cb7e0 2024-04-25T11:38:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.212+0000 7fc9c4908640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fc9c015eb20 con 0x7fc9c00b6210 2024-04-25T11:38:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.212+0000 7fc9c4908640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c00d1040 con 0x7fc9c0154f50 2024-04-25T11:38:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.213+0000 7fc9be59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c0154f50 0x7fc9c00c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.213+0000 7fc9bdd9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9c00b6210 0x7fc9c00cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.213+0000 7fc9be59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c0154f50 0x7fc9c00c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:37708/0 (socket says 172.21.15.40:37708) 2024-04-25T11:38:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.213+0000 7fc9be59c640 1 -- 172.21.15.40:0/2555269027 learned_addr learned my addr 172.21.15.40:0/2555269027 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.213+0000 7fc9be59c640 1 -- 172.21.15.40:0/2555269027 >> v2:172.21.15.62:3300/0 conn(0x7fc9c00b6210 msgr2=0x7fc9c00cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.213+0000 7fc9be59c640 1 --2- 172.21.15.40:0/2555269027 >> v2:172.21.15.62:3300/0 conn(0x7fc9c00b6210 0x7fc9c00cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.213+0000 7fc9be59c640 1 -- 172.21.15.40:0/2555269027 >> v1:172.21.15.62:6789/0 conn(0x7fc9c00cb7e0 legacy=0x7fc9c00c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.213+0000 7fc9be59c640 1 -- 172.21.15.40:0/2555269027 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c00cc150 con 0x7fc9c0154f50 2024-04-25T11:38:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.213+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/2555269027 >> v2:172.21.15.62:3300/0 conn(0x7fc9c00b6210 0x7fc9c00cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.214+0000 7fc9be59c640 1 --2- 172.21.15.40:0/2555269027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c0154f50 0x7fc9c00c7a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009870 tx=0x7fc9b802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d14b78dc1ec400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.214+0000 7fc9bd59a640 1 -- 172.21.15.40:0/2555269027 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8037070 con 0x7fc9c0154f50 2024-04-25T11:38:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.214+0000 7fc9bd59a640 1 -- 172.21.15.40:0/2555269027 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b8032440 con 0x7fc9c0154f50 2024-04-25T11:38:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.214+0000 7fc9bd59a640 1 -- 172.21.15.40:0/2555269027 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8032810 con 0x7fc9c0154f50 2024-04-25T11:38:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.214+0000 7fc9c4908640 1 -- 172.21.15.40:0/2555269027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c0154f50 msgr2=0x7fc9c00c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.214+0000 7fc9c4908640 1 --2- 172.21.15.40:0/2555269027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c0154f50 0x7fc9c00c7a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009870 tx=0x7fc9b802a1c0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.215+0000 7fc9c4908640 1 -- 172.21.15.40:0/2555269027 shutdown_connections 2024-04-25T11:38:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.215+0000 7fc9c4908640 1 --2- 172.21.15.40:0/2555269027 >> v2:172.21.15.62:3300/0 conn(0x7fc9c00b6210 0x7fc9c00cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.215+0000 7fc9c4908640 1 --2- 172.21.15.40:0/2555269027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c0154f50 0x7fc9c00c7a70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.215+0000 7fc9c4908640 1 -- 172.21.15.40:0/2555269027 >> 172.21.15.40:0/2555269027 conn(0x7fc9c00d0440 msgr2=0x7fc9c00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.215+0000 7fc9c4908640 1 -- 172.21.15.40:0/2555269027 shutdown_connections 2024-04-25T11:38:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.215+0000 7fc9c4908640 1 -- 172.21.15.40:0/2555269027 wait complete. 2024-04-25T11:38:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.216+0000 7fc9c4908640 1 Processor -- start 2024-04-25T11:38:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.216+0000 7fc9c4908640 1 -- start start 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9c4908640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c00cb7e0 0x7fc9c0150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9c4908640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9c0154f50 0x7fc9c01f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9c4908640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9c00c3b90 0x7fc9c01f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9c4908640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c00cc3f0 con 0x7fc9c00c3b90 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9c4908640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c015ed80 con 0x7fc9c0154f50 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9c4908640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c00c8170 con 0x7fc9c00cb7e0 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9be59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c00cb7e0 0x7fc9c0150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9be59c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c00cb7e0 0x7fc9c0150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:37724/0 (socket says 172.21.15.40:37724) 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9bdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9c0154f50 0x7fc9c01f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9be59c640 1 -- 172.21.15.40:0/851437506 learned_addr learned my addr 172.21.15.40:0/851437506 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9c00c3b90 0x7fc9c01f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9bed9d640 1 -- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c00cb7e0 msgr2=0x7fc9c0150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c00cb7e0 0x7fc9c0150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9bed9d640 1 -- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9c0154f50 msgr2=0x7fc9c01f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9c0154f50 0x7fc9c01f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9bed9d640 1 -- 172.21.15.40:0/851437506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c00ce720 con 0x7fc9c00c3b90 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9be59c640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c00cb7e0 0x7fc9c0150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9c0154f50 0x7fc9c01f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9c00c3b90 0x7fc9c01f4460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000c760 tx=0x7fc9b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9aeffd640 1 -- 172.21.15.40:0/851437506 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0016020 con 0x7fc9c00c3b90 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9aeffd640 1 -- 172.21.15.40:0/851437506 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b0006d50 con 0x7fc9c00c3b90 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.217+0000 7fc9c4908640 1 -- 172.21.15.40:0/851437506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c00c5290 con 0x7fc9c00c3b90 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.218+0000 7fc9aeffd640 1 -- 172.21.15.40:0/851437506 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0003bb0 con 0x7fc9c00c3b90 2024-04-25T11:38:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.218+0000 7fc9c4908640 1 -- 172.21.15.40:0/851437506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c00c40d0 con 0x7fc9c00c3b90 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.219+0000 7fc9aeffd640 1 -- 172.21.15.40:0/851437506 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9b00050d0 con 0x7fc9c00c3b90 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.219+0000 7fc9aeffd640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc9a006c920 0x7fc9a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.220+0000 7fc9aeffd640 1 -- 172.21.15.40:0/851437506 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc9b008b5b0 con 0x7fc9c00c3b90 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.220+0000 7fc9be59c640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc9a006c920 0x7fc9a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.220+0000 7fc9be59c640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc9a006c920 0x7fc9a006ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8035040 tx=0x7fc9b802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.220+0000 7fc9c7623740 1 Processor -- start 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9c7623740 1 -- start start 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abf690 0x5579b5b03a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abefc0 0x5579b5ac2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b0d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9c7623740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5579b5a28c50 con 0x5579b5abfa70 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9c7623740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5579b59c5830 con 0x5579b5abefc0 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9c7623740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5579b5a24c10 con 0x5579b5abf690 2024-04-25T11:38:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b0d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b0d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34736/0 (socket says 172.21.15.40:34736) 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9bed9d640 1 -- 172.21.15.40:0/324345175 learned_addr learned my addr 172.21.15.40:0/324345175 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/324345175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abefc0 0x5579b5ac2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.221+0000 7fc9be59c640 1 --2- 172.21.15.40:0/324345175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abf690 0x5579b5b03a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.222+0000 7fc9bed9d640 1 -- 172.21.15.40:0/324345175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abf690 msgr2=0x5579b5b03a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.222+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/324345175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abf690 0x5579b5b03a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.222+0000 7fc9bed9d640 1 -- 172.21.15.40:0/324345175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abefc0 msgr2=0x5579b5ac2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.222+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/324345175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abefc0 0x5579b5ac2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.222+0000 7fc9bed9d640 1 -- 172.21.15.40:0/324345175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b573b2d0 con 0x5579b5abfa70 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.222+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/324345175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b0d8a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0089b10 tx=0x7fc9b0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ba1a16f728ad02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.222+0000 7fc99ffff640 1 -- 172.21.15.40:0/324345175 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00576a0 con 0x5579b5abfa70 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.222+0000 7fc99ffff640 1 -- 172.21.15.40:0/324345175 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b0055a10 con 0x5579b5abfa70 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.222+0000 7fc99ffff640 1 -- 172.21.15.40:0/324345175 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0042d80 con 0x5579b5abfa70 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 -- 172.21.15.40:0/324345175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 msgr2=0x5579b5b0d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 --2- 172.21.15.40:0/324345175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b0d8a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0089b10 tx=0x7fc9b0000f40 comp rx=0 tx=0).stop 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 -- 172.21.15.40:0/324345175 shutdown_connections 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 --2- 172.21.15.40:0/324345175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b0d8a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 --2- 172.21.15.40:0/324345175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abefc0 0x5579b5ac2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 --2- 172.21.15.40:0/324345175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abf690 0x5579b5b03a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 -- 172.21.15.40:0/324345175 >> 172.21.15.40:0/324345175 conn(0x5579b5ac0c80 msgr2=0x5579b5ac3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 -- 172.21.15.40:0/324345175 shutdown_connections 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 -- 172.21.15.40:0/324345175 wait complete. 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 -- start start 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.223+0000 7fc9c7623740 1 Processor -- start 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 0x5579b5ae42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b06590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5ae0a90 0x5579b5ae0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9c7623740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5579b5b04700 con 0x5579b5abfa70 2024-04-25T11:38:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9c7623740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5579b59c3470 con 0x5579b5abf690 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9c7623740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5579b5ac20e0 con 0x5579b5ae0a90 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9c7623740 10 client.0 _ll_register_callbacks cb 0x5579b59e8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9bdd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b06590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9bdd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b06590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34752/0 (socket says 172.21.15.40:34752) 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9bdd9b640 1 -- 172.21.15.40:0/1889366705 learned_addr learned my addr 172.21.15.40:0/1889366705 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9be59c640 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 0x5579b5ae42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.224+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5ae0a90 0x5579b5ae0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9bdd9b640 1 -- 172.21.15.40:0/1889366705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5ae0a90 msgr2=0x5579b5ae0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5ae0a90 0x5579b5ae0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9bdd9b640 1 -- 172.21.15.40:0/1889366705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 msgr2=0x5579b5ae42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 0x5579b5ae42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9bdd9b640 1 -- 172.21.15.40:0/1889366705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b58056b0 con 0x5579b5abfa70 2024-04-25T11:38:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b06590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9c01519d0 tx=0x7fc9b4014dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4006ec7c0d323a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc99d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc99d7fa640 1 -- 172.21.15.40:0/1889366705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4004300 con 0x5579b5abfa70 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc99d7fa640 1 -- 172.21.15.40:0/1889366705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc9b4012850 con 0x5579b5abfa70 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9c7623740 1 -- 172.21.15.40:0/1889366705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b0055a10 con 0x5579b5abfa70 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc99d7fa640 1 -- 172.21.15.40:0/1889366705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc9b4027050 con 0x5579b5abfa70 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9c7623740 10 client.4719 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9c7623740 1 -- 172.21.15.40:0/1889366705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5579b5ae0550 con 0x5579b5abfa70 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc99d7fa640 1 -- 172.21.15.40:0/1889366705 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b0055a10 con 0x5579b5abfa70 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc99d7fa640 1 -- 172.21.15.40:0/1889366705 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fc9b402a020 con 0x5579b5abfa70 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9c7623740 10 client.4719 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9c7623740 10 client.4719 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9c7623740 20 client.4719 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.225+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] conn(0x5579b5b228a0 0x5579b5b24c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.226+0000 7fc9c7623740 4 client.4719 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.226+0000 7fc9c7623740 1 -- 172.21.15.40:0/1889366705 --> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579b5a36670 con 0x5579b5b228a0 2024-04-25T11:38:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.226+0000 7fc9be59c640 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] conn(0x5579b5b228a0 0x5579b5b24c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.226+0000 7fc9be59c640 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] conn(0x5579b5b228a0 0x5579b5b24c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7cc70985eb69d96 server_cookie=eff4a8888efa593a in_seq=0 out_seq=0 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.226+0000 7fc99d7fa640 10 client.4719 ms_handle_connect on v2:172.21.15.40:6832/4104143657 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc99d7fa640 1 -- 172.21.15.40:0/1889366705 <== mds.3 v2:172.21.15.40:6832/4104143657 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579b5a36670 con 0x5579b5b228a0 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc99d7fa640 10 client.4719 handle_command_reply: tid=0 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 client.4719 shutdown 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 -- 172.21.15.40:0/1889366705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 msgr2=0x5579b5b06590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b06590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9c01519d0 tx=0x7fc9b4014dd0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 -- 172.21.15.40:0/1889366705 shutdown_connections 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] conn(0x5579b5b228a0 0x5579b5b24c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5ae0a90 0x5579b5ae0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abfa70 0x5579b5b06590 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1889366705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 0x5579b5ae42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc99d7fa640 0 client.4719 ms_handle_reset on v2:172.21.15.40:6832/4104143657 2024-04-25T11:38:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 -- 172.21.15.40:0/1889366705 >> 172.21.15.40:0/1889366705 conn(0x5579b5abefc0 msgr2=0x5579b5abe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 -- 172.21.15.40:0/1889366705 shutdown_connections 2024-04-25T11:38:59.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 1 -- 172.21.15.40:0/1889366705 wait complete. 2024-04-25T11:38:59.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.227+0000 7fc9c7623740 20 client.4719 trim_cache size 0 max 16384 2024-04-25T11:38:59.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9c7623740 1 Processor -- start 2024-04-25T11:38:59.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9c7623740 1 -- start start 2024-04-25T11:38:59.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 0x5579b5ae8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 0x5579b5ade110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5ae0a90 0x5579b5ade650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9c7623740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01fd310 con 0x5579b5ae0a90 2024-04-25T11:38:59.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9c7623740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01567d0 con 0x5579b5abf690 2024-04-25T11:38:59.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9c7623740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5579b5b07020 con 0x5579b5abfa70 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5ae0a90 0x5579b5ade650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5ae0a90 0x5579b5ade650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34762/0 (socket says 172.21.15.40:34762) 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9bed9d640 1 -- 172.21.15.40:0/1163965055 learned_addr learned my addr 172.21.15.40:0/1163965055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9be59c640 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 0x5579b5ae8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.246+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 0x5579b5ade110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9bed9d640 1 -- 172.21.15.40:0/1163965055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 msgr2=0x5579b5ade110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 0x5579b5ade110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9bed9d640 1 -- 172.21.15.40:0/1163965055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 msgr2=0x5579b5ae8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 0x5579b5ae8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9bed9d640 1 -- 172.21.15.40:0/1163965055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b5b03a70 con 0x5579b5ae0a90 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9be59c640 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 0x5579b5ae8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5ae0a90 0x5579b5ade650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00590a0 tx=0x7fc9b00592a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c077eb45009807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc99f7fe640 1 -- 172.21.15.40:0/1163965055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0055a10 con 0x5579b5ae0a90 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc99f7fe640 1 -- 172.21.15.40:0/1163965055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b00770c0 con 0x5579b5ae0a90 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc99f7fe640 1 -- 172.21.15.40:0/1163965055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0042b60 con 0x5579b5ae0a90 2024-04-25T11:38:59.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9c7623740 1 -- 172.21.15.40:0/1163965055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5ae0a90 msgr2=0x5579b5ade650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5ae0a90 0x5579b5ade650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00590a0 tx=0x7fc9b00592a0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.247+0000 7fc9c7623740 1 -- 172.21.15.40:0/1163965055 shutdown_connections 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.248+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5ae0a90 0x5579b5ade650 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.248+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 0x5579b5ade110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.248+0000 7fc9c7623740 1 --2- 172.21.15.40:0/1163965055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5abf690 0x5579b5ae8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.248+0000 7fc9c7623740 1 -- 172.21.15.40:0/1163965055 >> 172.21.15.40:0/1163965055 conn(0x5579b5abefc0 msgr2=0x5579b5ac4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.248+0000 7fc9c7623740 1 -- 172.21.15.40:0/1163965055 shutdown_connections 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.248+0000 7fc9c7623740 1 -- 172.21.15.40:0/1163965055 wait complete. 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.248+0000 7fc9c7623740 1 -- start start 2024-04-25T11:38:59.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.248+0000 7fc9c7623740 1 Processor -- start 2024-04-25T11:38:59.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abf690 0x5579b5b832a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 0x5579b5b08330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9c7623740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5ae0a90 0x5579b5aed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9c7623740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5579b5b07e50 con 0x5579b5abf690 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9c7623740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5579b5ae0030 con 0x5579b5ae0a90 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9c7623740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5579b5adb310 con 0x5579b5abfa70 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9c7623740 10 client.0 _ll_register_callbacks cb 0x7fc9b40094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9be59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abf690 0x5579b5b832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9bed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5ae0a90 0x5579b5aed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9be59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abf690 0x5579b5b832a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34772/0 (socket says 172.21.15.40:34772) 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9be59c640 1 -- 172.21.15.40:0/2291972677 learned_addr learned my addr 172.21.15.40:0/2291972677 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 0x5579b5b08330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9be59c640 1 -- 172.21.15.40:0/2291972677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 msgr2=0x5579b5b08330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9be59c640 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 0x5579b5b08330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9be59c640 1 -- 172.21.15.40:0/2291972677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5ae0a90 msgr2=0x5579b5aed080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9be59c640 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5ae0a90 0x5579b5aed080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9be59c640 1 -- 172.21.15.40:0/2291972677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b5aed940 con 0x5579b5abf690 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.249+0000 7fc9bed9d640 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5ae0a90 0x5579b5aed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9be59c640 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abf690 0x5579b5b832a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8032e90 tx=0x7fc9b802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e1f7a877102899 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc99d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc99d7fa640 1 -- 172.21.15.40:0/2291972677 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8037040 con 0x5579b5abf690 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc99d7fa640 1 -- 172.21.15.40:0/2291972677 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc9b805aae0 con 0x5579b5abf690 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9c7623740 1 -- 172.21.15.40:0/2291972677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b00770c0 con 0x5579b5abf690 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc99d7fa640 1 -- 172.21.15.40:0/2291972677 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc9b805ace0 con 0x5579b5abf690 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9c7623740 10 client.4725 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9c7623740 1 -- 172.21.15.40:0/2291972677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5579b5b839f0 con 0x5579b5abf690 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc99d7fa640 1 -- 172.21.15.40:0/2291972677 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b00770c0 con 0x5579b5abf690 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc99d7fa640 1 -- 172.21.15.40:0/2291972677 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fc9b80502c0 con 0x5579b5abf690 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9c7623740 10 client.4725 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:59.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9c7623740 10 client.4725 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9c7623740 20 client.4725 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9c7623740 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] conn(0x5579b5ad7fa0 0x5579b5b8c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9c7623740 4 client.4725 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9c7623740 1 -- 172.21.15.40:0/2291972677 --> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] -- command(tid 0: {"prefix": "status"}) -- 0x5579b5ad1dc0 con 0x5579b5ad7fa0 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.250+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] conn(0x5579b5ad7fa0 0x5579b5b8c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc99d7fa640 10 client.4725 ms_handle_connect on v2:172.21.15.40:6832/4104143657 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9bdd9b640 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] conn(0x5579b5ad7fa0 0x5579b5b8c500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b017ba9d0b7838f server_cookie=b9c0474f1614d58f in_seq=0 out_seq=0 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc99d7fa640 1 -- 172.21.15.40:0/2291972677 <== mds.3 v2:172.21.15.40:6832/4104143657 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5579b5ad1dc0 con 0x5579b5ad7fa0 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc99d7fa640 10 client.4725 handle_command_reply: tid=0 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 client.4725 shutdown 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 -- 172.21.15.40:0/2291972677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abf690 msgr2=0x5579b5b832a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abf690 0x5579b5b832a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8032e90 tx=0x7fc9b802c070 comp rx=0 tx=0).stop 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 -- 172.21.15.40:0/2291972677 shutdown_connections 2024-04-25T11:38:59.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.40:6832/4104143657,v1:172.21.15.40:6835/4104143657] conn(0x5579b5ad7fa0 0x5579b5b8c500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579b5ae0a90 0x5579b5aed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5579b5abfa70 0x5579b5b08330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 --2- 172.21.15.40:0/2291972677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5579b5abf690 0x5579b5b832a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc99d7fa640 0 client.4725 ms_handle_reset on v2:172.21.15.40:6832/4104143657 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 -- 172.21.15.40:0/2291972677 >> 172.21.15.40:0/2291972677 conn(0x5579b5abefc0 msgr2=0x5579b5ac3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 -- 172.21.15.40:0/2291972677 shutdown_connections 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 1 -- 172.21.15.40:0/2291972677 wait complete. 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.251+0000 7fc9c7623740 20 client.4725 trim_cache size 0 max 16384 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "6d14dde7-764d-42d0-be37-c790a704858d", 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 3, 2024-04-25T11:38:59.244 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4260, 2024-04-25T11:38:59.245 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-25T11:38:59.245 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-25T11:38:59.245 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-25T11:38:59.245 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 17.235137657999999, 2024-04-25T11:38:59.245 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 17, 2024-04-25T11:38:59.245 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 20, 2024-04-25T11:38:59.245 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T11:38:59.245 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 39.998397308999998 2024-04-25T11:38:59.245 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:38:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 -- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc9a006c920 msgr2=0x7fc9a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc9a006c920 0x7fc9a006ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8035040 tx=0x7fc9b802e000 comp rx=0 tx=0).stop 2024-04-25T11:38:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 -- 172.21.15.40:0/851437506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9c00c3b90 msgr2=0x7fc9c01f4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9c00c3b90 0x7fc9c01f4460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000c760 tx=0x7fc9b000cc20 comp rx=0 tx=0).stop 2024-04-25T11:38:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 -- 172.21.15.40:0/851437506 shutdown_connections 2024-04-25T11:38:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc9a006c920 0x7fc9a006ede0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9c00c3b90 0x7fc9c01f4460 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9c0154f50 0x7fc9c01f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 --2- 172.21.15.40:0/851437506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9c00cb7e0 0x7fc9c0150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 -- 172.21.15.40:0/851437506 >> 172.21.15.40:0/851437506 conn(0x7fc9c00d0440 msgr2=0x7fc9c00cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 -- 172.21.15.40:0/851437506 shutdown_connections 2024-04-25T11:38:59.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.252+0000 7fc9c4908640 1 -- 172.21.15.40:0/851437506 wait complete. 2024-04-25T11:38:59.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:38:59.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.364+0000 7ff497c72640 1 Processor -- start 2024-04-25T11:38:59.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.364+0000 7ff497c72640 1 -- start start 2024-04-25T11:38:59.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff497c72640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4900c5130 0x7ff4901589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff497c72640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff490071cc0 con 0x7ff490158f20 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff497c72640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900c96a0 con 0x7ff490152e50 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff497c72640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff490071720 con 0x7ff4900c5130 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff49620f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff490158f20 0x7ff4900c3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37636/0 (socket says 172.21.15.40:37636) 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff49620f640 1 -- 172.21.15.40:0/159418385 learned_addr learned my addr 172.21.15.40:0/159418385 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff49520d640 1 --2- 172.21.15.40:0/159418385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4900c5130 0x7ff4901589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff494a0c640 1 -- 172.21.15.40:0/159418385 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 882890350 0 0) 0x7ff490071cc0 con 0x7ff490158f20 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff494a0c640 1 -- 172.21.15.40:0/159418385 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff47c003520 con 0x7ff490158f20 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff494a0c640 1 -- 172.21.15.40:0/159418385 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3873788442 0 0) 0x7ff4900c96a0 con 0x7ff490152e50 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff494a0c640 1 -- 172.21.15.40:0/159418385 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff490071cc0 con 0x7ff490152e50 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff494a0c640 1 -- 172.21.15.40:0/159418385 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 289988587 0 0) 0x7ff47c003520 con 0x7ff490158f20 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff49520d640 1 -- 172.21.15.40:0/159418385 >> v1:172.21.15.62:6789/0 conn(0x7ff490152e50 legacy=0x7ff4900c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff49520d640 1 -- 172.21.15.40:0/159418385 >> v1:172.21.15.40:6789/0 conn(0x7ff490158f20 legacy=0x7ff4900c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.365+0000 7ff49520d640 1 -- 172.21.15.40:0/159418385 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900719e0 con 0x7ff4900c5130 2024-04-25T11:38:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.366+0000 7ff49520d640 1 --2- 172.21.15.40:0/159418385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4900c5130 0x7ff4901589e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4800097d0 tx=0x7ff48002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3184b14d3717640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.366+0000 7ff494a0c640 1 -- 172.21.15.40:0/159418385 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480037070 con 0x7ff4900c5130 2024-04-25T11:38:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.366+0000 7ff494a0c640 1 -- 172.21.15.40:0/159418385 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48002ab90 con 0x7ff4900c5130 2024-04-25T11:38:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.366+0000 7ff494a0c640 1 -- 172.21.15.40:0/159418385 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4880019f0 con 0x7ff4900c5130 2024-04-25T11:38:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.367+0000 7ff497c72640 1 -- 172.21.15.40:0/159418385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4900c5130 msgr2=0x7ff4901589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.367+0000 7ff497c72640 1 --2- 172.21.15.40:0/159418385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4900c5130 0x7ff4901589e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4800097d0 tx=0x7ff48002a100 comp rx=0 tx=0).stop 2024-04-25T11:38:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.367+0000 7ff497c72640 1 -- 172.21.15.40:0/159418385 shutdown_connections 2024-04-25T11:38:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.367+0000 7ff497c72640 1 --2- 172.21.15.40:0/159418385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4900c5130 0x7ff4901589e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.367+0000 7ff497c72640 1 -- 172.21.15.40:0/159418385 >> 172.21.15.40:0/159418385 conn(0x7ff49006b970 msgr2=0x7ff49006f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.367+0000 7ff497c72640 1 -- 172.21.15.40:0/159418385 shutdown_connections 2024-04-25T11:38:59.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.367+0000 7ff497c72640 1 -- 172.21.15.40:0/159418385 wait complete. 2024-04-25T11:38:59.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.368+0000 7ff497c72640 1 Processor -- start 2024-04-25T11:38:59.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.368+0000 7ff497c72640 1 -- start start 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff497c72640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900c5130 0x7ff4901efac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff497c72640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff490152e50 0x7ff490153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff497c72640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff490158f20 0x7ff490149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff497c72640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900ca140 con 0x7ff4900c5130 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff497c72640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900c9c20 con 0x7ff490158f20 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff497c72640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff490071300 con 0x7ff490152e50 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff495a0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900c5130 0x7ff4901efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff495a0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900c5130 0x7ff4901efac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34786/0 (socket says 172.21.15.40:34786) 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff495a0e640 1 -- 172.21.15.40:0/727712775 learned_addr learned my addr 172.21.15.40:0/727712775 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff49620f640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff490158f20 0x7ff490149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff49520d640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff490152e50 0x7ff490153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff495a0e640 1 -- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff490152e50 msgr2=0x7ff490153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff495a0e640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff490152e50 0x7ff490153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff495a0e640 1 -- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff490158f20 msgr2=0x7ff490149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff495a0e640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff490158f20 0x7ff490149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff495a0e640 1 -- 172.21.15.40:0/727712775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49014b040 con 0x7ff4900c5130 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff495a0e640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900c5130 0x7ff4901efac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00c460 tx=0x7ff48c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff4867fc640 1 -- 172.21.15.40:0/727712775 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0052d0 con 0x7ff4900c5130 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff4867fc640 1 -- 172.21.15.40:0/727712775 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c005490 con 0x7ff4900c5130 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff4867fc640 1 -- 172.21.15.40:0/727712775 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c012870 con 0x7ff4900c5130 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff497c72640 1 -- 172.21.15.40:0/727712775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49014b2e0 con 0x7ff4900c5130 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.369+0000 7ff497c72640 1 -- 172.21.15.40:0/727712775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901ef2c0 con 0x7ff4900c5130 2024-04-25T11:38:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.371+0000 7ff4867fc640 1 -- 172.21.15.40:0/727712775 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff48c012a10 con 0x7ff4900c5130 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.371+0000 7ff4867fc640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.371+0000 7ff4867fc640 1 -- 172.21.15.40:0/727712775 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff48c08b6d0 con 0x7ff4900c5130 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.371+0000 7ff49520d640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.372+0000 7ff49520d640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff480034000 tx=0x7ff48002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.372+0000 7ff49a983740 1 Processor -- start 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.372+0000 7ff49a983740 1 -- start start 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff49a983740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdf690 0x56050cc23a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff49a983740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdefc0 0x56050cbe2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff49a983740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc2d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff49a983740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56050cb48c50 con 0x56050cbdfa70 2024-04-25T11:38:59.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff49a983740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56050cae5830 con 0x56050cbdf690 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff49a983740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56050cb44c10 con 0x56050cbdefc0 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff49620f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc2d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff495a0e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdf690 0x56050cc23a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff49520d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdefc0 0x56050cbe2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff495a0e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdf690 0x56050cc23a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:55380/0 (socket says 172.21.15.40:55380) 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff495a0e640 1 -- 172.21.15.40:0/3172736410 learned_addr learned my addr 172.21.15.40:0/3172736410 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.373+0000 7ff49520d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdefc0 0x56050cbe2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:37794/0 (socket says 172.21.15.40:37794) 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff495a0e640 1 -- 172.21.15.40:0/3172736410 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdefc0 msgr2=0x56050cbe2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff495a0e640 1 --2- 172.21.15.40:0/3172736410 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdefc0 0x56050cbe2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff495a0e640 1 -- 172.21.15.40:0/3172736410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 msgr2=0x56050cc2d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff495a0e640 1 --2- 172.21.15.40:0/3172736410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc2d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff495a0e640 1 -- 172.21.15.40:0/3172736410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050c85b2d0 con 0x56050cbdf690 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff49620f640 1 --2- 172.21.15.40:0/3172736410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc2d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff495a0e640 1 --2- 172.21.15.40:0/3172736410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdf690 0x56050cc23a90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff48c089c10 tx=0x7ff48c0590e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39280244febab4df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff4737fe640 1 -- 172.21.15.40:0/3172736410 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c062050 con 0x56050cbdf690 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff4737fe640 1 -- 172.21.15.40:0/3172736410 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c057770 con 0x56050cbdf690 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff4737fe640 1 -- 172.21.15.40:0/3172736410 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c055ae0 con 0x56050cbdf690 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff49a983740 1 -- 172.21.15.40:0/3172736410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdf690 msgr2=0x56050cc23a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.374+0000 7ff49a983740 1 --2- 172.21.15.40:0/3172736410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdf690 0x56050cc23a90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff48c089c10 tx=0x7ff48c0590e0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.375+0000 7ff49a983740 1 -- 172.21.15.40:0/3172736410 shutdown_connections 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.375+0000 7ff49a983740 1 --2- 172.21.15.40:0/3172736410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc2d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.375+0000 7ff49a983740 1 --2- 172.21.15.40:0/3172736410 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdefc0 0x56050cbe2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.375+0000 7ff49a983740 1 --2- 172.21.15.40:0/3172736410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdf690 0x56050cc23a90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.375+0000 7ff49a983740 1 -- 172.21.15.40:0/3172736410 >> 172.21.15.40:0/3172736410 conn(0x56050cbe0c80 msgr2=0x56050cbe3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.375+0000 7ff49a983740 1 -- 172.21.15.40:0/3172736410 shutdown_connections 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.375+0000 7ff49a983740 1 -- 172.21.15.40:0/3172736410 wait complete. 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.375+0000 7ff49a983740 1 -- start start 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.375+0000 7ff49a983740 1 Processor -- start 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49a983740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc0cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49a983740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc04160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49a983740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 0x56050cc38610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49a983740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56050cc24700 con 0x56050cbdf690 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49a983740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56050cae3470 con 0x56050cbdfa70 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49a983740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56050cae5830 con 0x56050cc27000 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49a983740 10 client.0 _ll_register_callbacks cb 0x56050cb08f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff495a0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc0cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff495a0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc0cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34804/0 (socket says 172.21.15.40:34804) 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff495a0e640 1 -- 172.21.15.40:0/3864389558 learned_addr learned my addr 172.21.15.40:0/3864389558 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49520d640 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc04160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49620f640 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 0x56050cc38610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff495a0e640 1 -- 172.21.15.40:0/3864389558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 msgr2=0x56050cc38610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff495a0e640 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 0x56050cc38610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff495a0e640 1 -- 172.21.15.40:0/3864389558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 msgr2=0x56050cc04160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff495a0e640 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc04160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff495a0e640 1 -- 172.21.15.40:0/3864389558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050c9256b0 con 0x56050cbdf690 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.376+0000 7ff49520d640 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc04160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff495a0e640 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc0cc80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00ce20 tx=0x7ff48c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5932bf4cc16a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff470ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff470ff9640 1 -- 172.21.15.40:0/3864389558 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c076060 con 0x56050cbdf690 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff470ff9640 1 -- 172.21.15.40:0/3864389558 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff48c055ae0 con 0x56050cbdf690 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49a983740 1 -- 172.21.15.40:0/3864389558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48c057770 con 0x56050cbdf690 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff470ff9640 1 -- 172.21.15.40:0/3864389558 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff48c062050 con 0x56050cbdf690 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49a983740 10 client.4737 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49a983740 1 -- 172.21.15.40:0/3864389558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x56050cc28a60 con 0x56050cbdf690 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff470ff9640 1 -- 172.21.15.40:0/3864389558 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c057770 con 0x56050cbdf690 2024-04-25T11:38:59.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff470ff9640 1 -- 172.21.15.40:0/3864389558 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7ff48c061bc0 con 0x56050cbdf690 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49a983740 10 client.4737 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49a983740 10 client.4737 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49a983740 20 client.4737 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49a983740 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] conn(0x56050cc41940 0x56050cc43d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49a983740 4 client.4737 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49a983740 1 -- 172.21.15.40:0/3864389558 --> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050cb56670 con 0x56050cc41940 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.377+0000 7ff49520d640 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] conn(0x56050cc41940 0x56050cc43d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.378+0000 7ff49520d640 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] conn(0x56050cc41940 0x56050cc43d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aed376543eaff0cc server_cookie=adeed955804452d1 in_seq=0 out_seq=0 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.378+0000 7ff470ff9640 10 client.4737 ms_handle_connect on v2:172.21.15.40:6833/535060457 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.378+0000 7ff470ff9640 1 -- 172.21.15.40:0/3864389558 <== mds.2 v2:172.21.15.40:6833/535060457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56050cb56670 con 0x56050cc41940 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.378+0000 7ff470ff9640 10 client.4737 handle_command_reply: tid=0 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.378+0000 7ff49a983740 1 client.4737 shutdown 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 -- 172.21.15.40:0/3864389558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 msgr2=0x56050cc0cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc0cc80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00ce20 tx=0x7ff48c000ed0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 -- 172.21.15.40:0/3864389558 shutdown_connections 2024-04-25T11:38:59.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] conn(0x56050cc41940 0x56050cc43d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 0x56050cc38610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff470ff9640 0 client.4737 ms_handle_reset on v2:172.21.15.40:6833/535060457 2024-04-25T11:38:59.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc04160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 --2- 172.21.15.40:0/3864389558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc0cc80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 -- 172.21.15.40:0/3864389558 >> 172.21.15.40:0/3864389558 conn(0x56050cbdefc0 msgr2=0x56050cbde1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 -- 172.21.15.40:0/3864389558 shutdown_connections 2024-04-25T11:38:59.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 1 -- 172.21.15.40:0/3864389558 wait complete. 2024-04-25T11:38:59.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.379+0000 7ff49a983740 20 client.4737 trim_cache size 0 max 16384 2024-04-25T11:38:59.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.397+0000 7ff49a983740 1 Processor -- start 2024-04-25T11:38:59.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.397+0000 7ff49a983740 1 -- start start 2024-04-25T11:38:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49a983740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc9a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49a983740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc9a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49a983740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 0x56050cc0c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49a983740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff49014cab0 con 0x56050cbdf690 2024-04-25T11:38:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49a983740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56050cae5830 con 0x56050cbdfa70 2024-04-25T11:38:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49a983740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56050cc24700 con 0x56050cc27000 2024-04-25T11:38:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff495a0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc9a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff495a0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc9a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34806/0 (socket says 172.21.15.40:34806) 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff495a0e640 1 -- 172.21.15.40:0/4087714207 learned_addr learned my addr 172.21.15.40:0/4087714207 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49520d640 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc9a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff495a0e640 1 -- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 msgr2=0x56050cc0c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49620f640 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 0x56050cc0c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff495a0e640 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 0x56050cc0c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff495a0e640 1 -- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 msgr2=0x56050cc9a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff495a0e640 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc9a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff495a0e640 1 -- 172.21.15.40:0/4087714207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050cc235e0 con 0x56050cbdf690 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49520d640 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc9a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.398+0000 7ff49620f640 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 0x56050cc0c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff495a0e640 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc9a080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff48c059400 tx=0x7ff48c088000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6944182848a69e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff472ffd640 1 -- 172.21.15.40:0/4087714207 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c059470 con 0x56050cbdf690 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff472ffd640 1 -- 172.21.15.40:0/4087714207 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c057510 con 0x56050cbdf690 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff472ffd640 1 -- 172.21.15.40:0/4087714207 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c061bc0 con 0x56050cbdf690 2024-04-25T11:38:59.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff49a983740 1 -- 172.21.15.40:0/4087714207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 msgr2=0x56050cc9a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff49a983740 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc9a080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff48c059400 tx=0x7ff48c088000 comp rx=0 tx=0).stop 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff49a983740 1 -- 172.21.15.40:0/4087714207 shutdown_connections 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff49a983740 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cc27000 0x56050cc0c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff49a983740 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cbdfa70 0x56050cc9a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff49a983740 1 --2- 172.21.15.40:0/4087714207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdf690 0x56050cc9a080 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff49a983740 1 -- 172.21.15.40:0/4087714207 >> 172.21.15.40:0/4087714207 conn(0x56050cbdefc0 msgr2=0x56050cbd9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff49a983740 1 -- 172.21.15.40:0/4087714207 shutdown_connections 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.399+0000 7ff49a983740 1 -- 172.21.15.40:0/4087714207 wait complete. 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.400+0000 7ff49a983740 1 -- start start 2024-04-25T11:38:59.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.400+0000 7ff49a983740 1 Processor -- start 2024-04-25T11:38:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49a983740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdf690 0x56050cc31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49a983740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49a983740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cc27000 0x56050cc98ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49a983740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56050cc39130 con 0x56050cbdfa70 2024-04-25T11:38:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49a983740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56050cae3470 con 0x56050cc27000 2024-04-25T11:38:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49a983740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56050cc3fa70 con 0x56050cbdf690 2024-04-25T11:38:59.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49a983740 10 client.0 _ll_register_callbacks cb 0x7ff4880019f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49520d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49520d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc216b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34816/0 (socket says 172.21.15.40:34816) 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49520d640 1 -- 172.21.15.40:0/2335109116 learned_addr learned my addr 172.21.15.40:0/2335109116 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff495a0e640 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdf690 0x56050cc31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49620f640 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cc27000 0x56050cc98ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49520d640 1 -- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdf690 msgr2=0x56050cc31eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49520d640 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdf690 0x56050cc31eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49520d640 1 -- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cc27000 msgr2=0x56050cc98ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49520d640 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cc27000 0x56050cc98ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49520d640 1 -- 172.21.15.40:0/2335109116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050cc21bf0 con 0x56050cbdfa70 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff495a0e640 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdf690 0x56050cc31eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49620f640 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cc27000 0x56050cc98ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff470ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T11:38:59.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49520d640 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc216b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff48002ad10 tx=0x7ff480004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7453035b1ab848de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff470ff9640 1 -- 172.21.15.40:0/2335109116 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480037040 con 0x56050cbdfa70 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff470ff9640 1 -- 172.21.15.40:0/2335109116 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff48004c630 con 0x56050cbdfa70 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.401+0000 7ff49a983740 1 -- 172.21.15.40:0/2335109116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48c057510 con 0x56050cbdfa70 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff470ff9640 1 -- 172.21.15.40:0/2335109116 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff48005a050 con 0x56050cbdfa70 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff49a983740 10 client.4743 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff49a983740 1 -- 172.21.15.40:0/2335109116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x56050cbde420 con 0x56050cbdfa70 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff470ff9640 1 -- 172.21.15.40:0/2335109116 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c057510 con 0x56050cbdfa70 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff470ff9640 1 -- 172.21.15.40:0/2335109116 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7ff48005d020 con 0x56050cbdfa70 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff49a983740 10 client.4743 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff49a983740 10 client.4743 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff49a983740 20 client.4743 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff49a983740 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] conn(0x56050cc02c50 0x56050ccaba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff49a983740 4 client.4743 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff49a983740 1 -- 172.21.15.40:0/2335109116 --> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] -- command(tid 0: {"prefix": "status"}) -- 0x56050cb504e0 con 0x56050cc02c50 2024-04-25T11:38:59.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff495a0e640 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] conn(0x56050cc02c50 0x56050ccaba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff470ff9640 10 client.4743 ms_handle_connect on v2:172.21.15.40:6833/535060457 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.402+0000 7ff495a0e640 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] conn(0x56050cc02c50 0x56050ccaba30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d098cfaf7e763db0 server_cookie=5e0548206170e1ef in_seq=0 out_seq=0 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff470ff9640 1 -- 172.21.15.40:0/2335109116 <== mds.2 v2:172.21.15.40:6833/535060457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56050cb504e0 con 0x56050cc02c50 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff470ff9640 10 client.4743 handle_command_reply: tid=0 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 client.4743 shutdown 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 -- 172.21.15.40:0/2335109116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 msgr2=0x56050cc216b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc216b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff48002ad10 tx=0x7ff480004730 comp rx=0 tx=0).stop 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 -- 172.21.15.40:0/2335109116 shutdown_connections 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.40:6833/535060457,v1:172.21.15.40:6836/535060457] conn(0x56050cc02c50 0x56050ccaba30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56050cc27000 0x56050cc98ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56050cbdfa70 0x56050cc216b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 --2- 172.21.15.40:0/2335109116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56050cbdf690 0x56050cc31eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 -- 172.21.15.40:0/2335109116 >> 172.21.15.40:0/2335109116 conn(0x56050cbdefc0 msgr2=0x56050cc0c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff470ff9640 0 client.4743 ms_handle_reset on v2:172.21.15.40:6833/535060457 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 -- 172.21.15.40:0/2335109116 shutdown_connections 2024-04-25T11:38:59.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 1 -- 172.21.15.40:0/2335109116 wait complete. 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff49a983740 20 client.4743 trim_cache size 0 max 16384 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "6d14dde7-764d-42d0-be37-c790a704858d", 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 2, 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4279, 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 22.685061469000001, 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 17, 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 20, 2024-04-25T11:38:59.397 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T11:38:59.398 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 40.149832205999999 2024-04-25T11:38:59.398 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:38:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff497c72640 1 -- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff47406c8f0 msgr2=0x7ff47406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff497c72640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff480034000 tx=0x7ff48002e000 comp rx=0 tx=0).stop 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff497c72640 1 -- 172.21.15.40:0/727712775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900c5130 msgr2=0x7ff4901efac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff497c72640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900c5130 0x7ff4901efac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00c460 tx=0x7ff48c00c920 comp rx=0 tx=0).stop 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff497c72640 1 -- 172.21.15.40:0/727712775 shutdown_connections 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff497c72640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff497c72640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff490158f20 0x7ff490149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff497c72640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff490152e50 0x7ff490153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.403+0000 7ff497c72640 1 --2- 172.21.15.40:0/727712775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4900c5130 0x7ff4901efac0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.404+0000 7ff497c72640 1 -- 172.21.15.40:0/727712775 >> 172.21.15.40:0/727712775 conn(0x7ff49006b970 msgr2=0x7ff49006f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.404+0000 7ff497c72640 1 -- 172.21.15.40:0/727712775 shutdown_connections 2024-04-25T11:38:59.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.404+0000 7ff497c72640 1 -- 172.21.15.40:0/727712775 wait complete. 2024-04-25T11:38:59.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T11:38:59.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.515+0000 7f47ab564640 1 Processor -- start 2024-04-25T11:38:59.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.515+0000 7f47ab564640 1 -- start start 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.516+0000 7f47ab564640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 0x7f47a415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.516+0000 7f47ab564640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47a40c7030 0x7f47a40c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.516+0000 7f47ab564640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47a40c89e0 con 0x7f47a40c75f0 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.516+0000 7f47ab564640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f47a40c8790 con 0x7f47a40c7030 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.516+0000 7f47ab564640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f47a40c8cc0 con 0x7f47a4154c80 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.516+0000 7f47a9b01640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47a40c7030 0x7f47a40c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.516+0000 7f47a9b01640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47a40c7030 0x7f47a40c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34820/0 (socket says 172.21.15.40:34820) 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.516+0000 7f47a9b01640 1 -- 172.21.15.40:0/913854738 learned_addr learned my addr 172.21.15.40:0/913854738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.516+0000 7f47a9300640 1 --2- 172.21.15.40:0/913854738 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 0x7f47a415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.517+0000 7f47a9b01640 1 -- 172.21.15.40:0/913854738 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 msgr2=0x7f47a415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.517+0000 7f47a9b01640 1 --2- 172.21.15.40:0/913854738 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 0x7f47a415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.517+0000 7f47a9b01640 1 -- 172.21.15.40:0/913854738 >> v1:172.21.15.62:6789/0 conn(0x7f47a40c75f0 legacy=0x7f47a4070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.517+0000 7f47a9b01640 1 -- 172.21.15.40:0/913854738 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a40c8fa0 con 0x7f47a40c7030 2024-04-25T11:38:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.517+0000 7f47a9b01640 1 --2- 172.21.15.40:0/913854738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 0x7f47a40c7d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4794009e50 tx=0x7f479402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f307652788ea3f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.517+0000 7f47a3fff640 1 -- 172.21.15.40:0/913854738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794036070 con 0x7f47a40c7030 2024-04-25T11:38:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.517+0000 7f47a3fff640 1 -- 172.21.15.40:0/913854738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4794031600 con 0x7f47a40c7030 2024-04-25T11:38:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.517+0000 7f47a3fff640 1 -- 172.21.15.40:0/913854738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47940319d0 con 0x7f47a40c7030 2024-04-25T11:38:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.518+0000 7f47ab564640 1 -- 172.21.15.40:0/913854738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 msgr2=0x7f47a40c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.518+0000 7f47ab564640 1 --2- 172.21.15.40:0/913854738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 0x7f47a40c7d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4794009e50 tx=0x7f479402a7a0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.518+0000 7f47ab564640 1 -- 172.21.15.40:0/913854738 shutdown_connections 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.518+0000 7f47ab564640 1 --2- 172.21.15.40:0/913854738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 0x7f47a40c7d80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.518+0000 7f47ab564640 1 --2- 172.21.15.40:0/913854738 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 0x7f47a415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.518+0000 7f47ab564640 1 -- 172.21.15.40:0/913854738 >> 172.21.15.40:0/913854738 conn(0x7f47a40727a0 msgr2=0x7f47a4072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.518+0000 7f47ab564640 1 -- 172.21.15.40:0/913854738 shutdown_connections 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.518+0000 7f47ab564640 1 -- 172.21.15.40:0/913854738 wait complete. 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.519+0000 7f47ab564640 1 Processor -- start 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.519+0000 7f47ab564640 1 -- start start 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47ab564640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 0x7f47a414d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47ab564640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a40c75f0 0x7f47a4156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47ab564640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 0x7f47a4156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47ab564640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f47a40cd610 con 0x7f47a40c7030 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47ab564640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f47a40cd330 con 0x7f47a40c75f0 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47ab564640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f47a4155060 con 0x7f47a4154c80 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9300640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 0x7f47a414d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9300640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 0x7f47a414d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34826/0 (socket says 172.21.15.40:34826) 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9300640 1 -- 172.21.15.40:0/2555278885 learned_addr learned my addr 172.21.15.40:0/2555278885 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a8aff640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a40c75f0 0x7f47a4156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9b01640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 0x7f47a4156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9300640 1 -- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 msgr2=0x7f47a4156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9300640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 0x7f47a4156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9300640 1 -- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a40c75f0 msgr2=0x7f47a4156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9300640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a40c75f0 0x7f47a4156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9300640 1 -- 172.21.15.40:0/2555278885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a41f18a0 con 0x7f47a40c7030 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.520+0000 7f47a9300640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 0x7f47a414d7c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e8a0 tx=0x7f479c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.521+0000 7f47a1ffb640 1 -- 172.21.15.40:0/2555278885 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c017070 con 0x7f47a40c7030 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.521+0000 7f47a1ffb640 1 -- 172.21.15.40:0/2555278885 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f479c00c940 con 0x7f47a40c7030 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.521+0000 7f47a1ffb640 1 -- 172.21.15.40:0/2555278885 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c00cd30 con 0x7f47a40c7030 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.521+0000 7f47ab564640 1 -- 172.21.15.40:0/2555278885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a41527c0 con 0x7f47a40c7030 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.521+0000 7f47ab564640 1 -- 172.21.15.40:0/2555278885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a4152d80 con 0x7f47a40c7030 2024-04-25T11:38:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.522+0000 7f47a1ffb640 1 -- 172.21.15.40:0/2555278885 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f479c01a070 con 0x7f47a40c7030 2024-04-25T11:38:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.523+0000 7f47a1ffb640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f479006c8f0 0x7f479006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.523+0000 7f47a1ffb640 1 -- 172.21.15.40:0/2555278885 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f479c08b9c0 con 0x7f47a40c7030 2024-04-25T11:38:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.523+0000 7f47a8aff640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f479006c8f0 0x7f479006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.523+0000 7f47a8aff640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f479006c8f0 0x7f479006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f47a414e0f0 tx=0x7f4798009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.524+0000 7f47ae27e740 1 Processor -- start 2024-04-25T11:38:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.524+0000 7f47ae27e740 1 -- start start 2024-04-25T11:38:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.524+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 0x560b8bb55a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb10fc0 0x560b8bb14f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 0x560b8bb5f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47ae27e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560b8ba7ac50 con 0x560b8bb10fc0 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47ae27e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b8ba17830 con 0x560b8bb11a70 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47ae27e740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560b8ba76c10 con 0x560b8bb11690 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a8aff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb10fc0 0x560b8bb14f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a9300640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 0x560b8bb55a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a9b01640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 0x560b8bb5f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a8aff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb10fc0 0x560b8bb14f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34832/0 (socket says 172.21.15.40:34832) 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a8aff640 1 -- 172.21.15.40:0/694596258 learned_addr learned my addr 172.21.15.40:0/694596258 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a8aff640 1 -- 172.21.15.40:0/694596258 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 msgr2=0x560b8bb55a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a8aff640 1 --2- 172.21.15.40:0/694596258 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 0x560b8bb55a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a8aff640 1 -- 172.21.15.40:0/694596258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 msgr2=0x560b8bb5f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a8aff640 1 --2- 172.21.15.40:0/694596258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 0x560b8bb5f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a8aff640 1 -- 172.21.15.40:0/694596258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b8b78d2d0 con 0x560b8bb10fc0 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f47a8aff640 1 --2- 172.21.15.40:0/694596258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb10fc0 0x560b8bb14f40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f479800f980 tx=0x7f4798012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93058db257581a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f478affd640 1 -- 172.21.15.40:0/694596258 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798025070 con 0x560b8bb10fc0 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f478affd640 1 -- 172.21.15.40:0/694596258 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f479801b050 con 0x560b8bb10fc0 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.525+0000 7f478affd640 1 -- 172.21.15.40:0/694596258 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798014e00 con 0x560b8bb10fc0 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 -- 172.21.15.40:0/694596258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb10fc0 msgr2=0x560b8bb14f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 --2- 172.21.15.40:0/694596258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb10fc0 0x560b8bb14f40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f479800f980 tx=0x7f4798012000 comp rx=0 tx=0).stop 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 -- 172.21.15.40:0/694596258 shutdown_connections 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 --2- 172.21.15.40:0/694596258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 0x560b8bb5f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 --2- 172.21.15.40:0/694596258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb10fc0 0x560b8bb14f40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 --2- 172.21.15.40:0/694596258 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 0x560b8bb55a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 -- 172.21.15.40:0/694596258 >> 172.21.15.40:0/694596258 conn(0x560b8bb12c80 msgr2=0x560b8bb15580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 -- 172.21.15.40:0/694596258 shutdown_connections 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 -- 172.21.15.40:0/694596258 wait complete. 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.526+0000 7f47ae27e740 1 -- start start 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.527+0000 7f47ae27e740 1 Processor -- start 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.527+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 0x560b8bb389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.527+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 0x560b8bb3f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.527+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb321a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.527+0000 7f47ae27e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560b8bb56700 con 0x560b8bb6a530 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.527+0000 7f47ae27e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b8ba15470 con 0x560b8bb11a70 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.527+0000 7f47ae27e740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560b8bb140e0 con 0x560b8bb11690 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.527+0000 7f47ae27e740 10 client.0 _ll_register_callbacks cb 0x560b8ba3af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.527+0000 7f47a9b01640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a9300640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 0x560b8bb389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a8aff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 0x560b8bb3f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a9b01640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb321a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34846/0 (socket says 172.21.15.40:34846) 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a9b01640 1 -- 172.21.15.40:0/1643901319 learned_addr learned my addr 172.21.15.40:0/1643901319 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a8aff640 1 -- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 msgr2=0x560b8bb389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a8aff640 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 0x560b8bb389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a8aff640 1 -- 172.21.15.40:0/1643901319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 msgr2=0x560b8bb321a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a8aff640 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb321a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a8aff640 1 -- 172.21.15.40:0/1643901319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b8b8576b0 con 0x560b8bb11a70 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a9b01640 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb321a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47a8aff640 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 0x560b8bb3f480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4798022000 tx=0x7f4798000f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afc6aca6f9ca1ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f477ffff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f477ffff640 1 -- 172.21.15.40:0/1643901319 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798025040 con 0x560b8bb11a70 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47ae27e740 1 -- 172.21.15.40:0/1643901319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f479801b050 con 0x560b8bb11a70 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f477ffff640 1 -- 172.21.15.40:0/1643901319 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4798021a50 con 0x560b8bb11a70 2024-04-25T11:38:59.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f477ffff640 1 -- 172.21.15.40:0/1643901319 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4798038050 con 0x560b8bb11a70 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47ae27e740 10 client.4717 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.528+0000 7f47ae27e740 1 -- 172.21.15.40:0/1643901319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560b8bb16490 con 0x560b8bb11a70 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f477ffff640 1 -- 172.21.15.40:0/1643901319 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f479801b050 con 0x560b8bb11a70 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f477ffff640 1 -- 172.21.15.40:0/1643901319 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f479803b020 con 0x560b8bb11a70 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f47ae27e740 10 client.4717 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f47ae27e740 10 client.4717 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f47ae27e740 20 client.4717 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] conn(0x560b8bb60b60 0x560b8bb75f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f47ae27e740 4 client.4717 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f47ae27e740 1 -- 172.21.15.40:0/1643901319 --> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b8ba88670 con 0x560b8bb60b60 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f47a9300640 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] conn(0x560b8bb60b60 0x560b8bb75f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f47a9300640 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] conn(0x560b8bb60b60 0x560b8bb75f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8ee694a74ce59eb server_cookie=b00878db384dc20f in_seq=0 out_seq=0 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.529+0000 7f477ffff640 10 client.4717 ms_handle_connect on v2:172.21.15.62:6834/3758619338 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f477ffff640 1 -- 172.21.15.40:0/1643901319 <== mds.1 v2:172.21.15.62:6834/3758619338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b8ba88670 con 0x560b8bb60b60 2024-04-25T11:38:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f477ffff640 10 client.4717 handle_command_reply: tid=0 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 client.4717 shutdown 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 -- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 msgr2=0x560b8bb3f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 0x560b8bb3f480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4798022000 tx=0x7f4798000f20 comp rx=0 tx=0).stop 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 -- 172.21.15.40:0/1643901319 shutdown_connections 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] conn(0x560b8bb60b60 0x560b8bb75f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb321a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f477ffff640 0 client.4717 ms_handle_reset on v2:172.21.15.62:6834/3758619338 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11a70 0x560b8bb3f480 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1643901319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11690 0x560b8bb389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 -- 172.21.15.40:0/1643901319 >> 172.21.15.40:0/1643901319 conn(0x560b8bb10fc0 msgr2=0x560b8bb101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 -- 172.21.15.40:0/1643901319 shutdown_connections 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.530+0000 7f47ae27e740 1 -- 172.21.15.40:0/1643901319 wait complete. 2024-04-25T11:38:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.531+0000 7f47ae27e740 20 client.4717 trim_cache size 0 max 16384 2024-04-25T11:38:59.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.549+0000 7f47ae27e740 1 Processor -- start 2024-04-25T11:38:59.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.549+0000 7f47ae27e740 1 -- start start 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11690 0x560b8bb0be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb0c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb7afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47ae27e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41f3f40 con 0x560b8bb6a530 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47ae27e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b8bb16710 con 0x560b8bb11690 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47ae27e740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560b8bb39450 con 0x560b8bb11a70 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9b01640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb7afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9b01640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb7afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34848/0 (socket says 172.21.15.40:34848) 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9b01640 1 -- 172.21.15.40:0/2767299942 learned_addr learned my addr 172.21.15.40:0/2767299942 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a8aff640 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb0c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9300640 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11690 0x560b8bb0be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9b01640 1 -- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 msgr2=0x560b8bb0c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9b01640 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb0c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9b01640 1 -- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11690 msgr2=0x560b8bb0be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9b01640 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11690 0x560b8bb0be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9b01640 1 -- 172.21.15.40:0/2767299942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b8bb382a0 con 0x560b8bb6a530 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9300640 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11690 0x560b8bb0be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a8aff640 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb0c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f47a9b01640 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb7afa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f479403f500 tx=0x7f479402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafab4e5b70c7769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f478a7fc640 1 -- 172.21.15.40:0/2767299942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479404c070 con 0x560b8bb6a530 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f478a7fc640 1 -- 172.21.15.40:0/2767299942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4794037040 con 0x560b8bb6a530 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.550+0000 7f478a7fc640 1 -- 172.21.15.40:0/2767299942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794036040 con 0x560b8bb6a530 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.551+0000 7f47ae27e740 1 -- 172.21.15.40:0/2767299942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 msgr2=0x560b8bb7afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.551+0000 7f47ae27e740 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb7afa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f479403f500 tx=0x7f479402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.551+0000 7f47ae27e740 1 -- 172.21.15.40:0/2767299942 shutdown_connections 2024-04-25T11:38:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.551+0000 7f47ae27e740 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb6a530 0x560b8bb7afa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.551+0000 7f47ae27e740 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb0c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.551+0000 7f47ae27e740 1 --2- 172.21.15.40:0/2767299942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb11690 0x560b8bb0be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.551+0000 7f47ae27e740 1 -- 172.21.15.40:0/2767299942 >> 172.21.15.40:0/2767299942 conn(0x560b8bb10fc0 msgr2=0x560b8bb50850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.551+0000 7f47ae27e740 1 -- 172.21.15.40:0/2767299942 shutdown_connections 2024-04-25T11:38:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.551+0000 7f47ae27e740 1 -- 172.21.15.40:0/2767299942 wait complete. 2024-04-25T11:38:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.552+0000 7f47ae27e740 1 -- start start 2024-04-25T11:38:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.552+0000 7f47ae27e740 1 Processor -- start 2024-04-25T11:38:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.552+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb11690 0x560b8bb53d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.552+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb5fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.552+0000 7f47ae27e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb6a530 0x560b8bb3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.552+0000 7f47ae27e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560b8bb609e0 con 0x560b8bb11690 2024-04-25T11:38:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.552+0000 7f47ae27e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b8bb32bb0 con 0x560b8bb6a530 2024-04-25T11:38:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.552+0000 7f47ae27e740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560b8bb61030 con 0x560b8bb11a70 2024-04-25T11:38:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47ae27e740 10 client.0 _ll_register_callbacks cb 0x7f4798023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9300640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb11690 0x560b8bb53d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9300640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb11690 0x560b8bb53d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34850/0 (socket says 172.21.15.40:34850) 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9300640 1 -- 172.21.15.40:0/1662144016 learned_addr learned my addr 172.21.15.40:0/1662144016 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a8aff640 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb5fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9b01640 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb6a530 0x560b8bb3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9300640 1 -- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 msgr2=0x560b8bb5fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9300640 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb5fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9300640 1 -- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb6a530 msgr2=0x560b8bb3bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9300640 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb6a530 0x560b8bb3bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9300640 1 -- 172.21.15.40:0/1662144016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b8bb50ed0 con 0x560b8bb11690 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a8aff640 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb5fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9b01640 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb6a530 0x560b8bb3bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47a9300640 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb11690 0x560b8bb53d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f479c059410 tx=0x7f479c059890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2411d8ed8f2ae573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f477ffff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T11:38:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f477ffff640 1 -- 172.21.15.40:0/1662144016 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c070040 con 0x560b8bb11690 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f477ffff640 1 -- 172.21.15.40:0/1662144016 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f479c057a10 con 0x560b8bb11690 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47ae27e740 1 -- 172.21.15.40:0/1662144016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4794037040 con 0x560b8bb11690 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f477ffff640 1 -- 172.21.15.40:0/1662144016 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f479c021040 con 0x560b8bb11690 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47ae27e740 10 client.4761 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f47ae27e740 1 -- 172.21.15.40:0/1662144016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560b8bb70e30 con 0x560b8bb11690 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f477ffff640 1 -- 172.21.15.40:0/1662144016 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4794037040 con 0x560b8bb11690 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.553+0000 7f477ffff640 1 -- 172.21.15.40:0/1662144016 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f479c024060 con 0x560b8bb11690 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 10 client.4761 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 10 client.4761 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 20 client.4761 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] conn(0x560b8bbcf3b0 0x560b8bbcf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 4 client.4761 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 -- 172.21.15.40:0/1662144016 --> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] -- command(tid 0: {"prefix": "status"}) -- 0x560b8ba824e0 con 0x560b8bbcf3b0 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47a8aff640 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] conn(0x560b8bbcf3b0 0x560b8bbcf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47a8aff640 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] conn(0x560b8bbcf3b0 0x560b8bbcf790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e797dd75a86047b server_cookie=6f2caa0efdf08f9e in_seq=0 out_seq=0 2024-04-25T11:38:59.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f477ffff640 10 client.4761 ms_handle_connect on v2:172.21.15.62:6834/3758619338 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f477ffff640 1 -- 172.21.15.40:0/1662144016 <== mds.1 v2:172.21.15.62:6834/3758619338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560b8ba824e0 con 0x560b8bbcf3b0 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f477ffff640 10 client.4761 handle_command_reply: tid=0 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 client.4761 shutdown 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 -- 172.21.15.40:0/1662144016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb11690 msgr2=0x560b8bb53d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb11690 0x560b8bb53d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f479c059410 tx=0x7f479c059890 comp rx=0 tx=0).stop 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 -- 172.21.15.40:0/1662144016 shutdown_connections 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:6834/3758619338,v1:172.21.15.62:6835/3758619338] conn(0x560b8bbcf3b0 0x560b8bbcf790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b8bb6a530 0x560b8bb3bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560b8bb11a70 0x560b8bb5fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f477ffff640 0 client.4761 ms_handle_reset on v2:172.21.15.62:6834/3758619338 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 --2- 172.21.15.40:0/1662144016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560b8bb11690 0x560b8bb53d40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.554+0000 7f47ae27e740 1 -- 172.21.15.40:0/1662144016 >> 172.21.15.40:0/1662144016 conn(0x560b8bb10fc0 msgr2=0x560b8bb50640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ae27e740 1 -- 172.21.15.40:0/1662144016 shutdown_connections 2024-04-25T11:38:59.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ae27e740 1 -- 172.21.15.40:0/1662144016 wait complete. 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ae27e740 20 client.4761 trim_cache size 0 max 16384 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "6d14dde7-764d-42d0-be37-c790a704858d", 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 1, 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4294, 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 28.862691956999999, 2024-04-25T11:38:59.549 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 17, 2024-04-25T11:38:59.550 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 20, 2024-04-25T11:38:59.550 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T11:38:59.550 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 40.283646306999998 2024-04-25T11:38:59.550 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 -- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f479006c8f0 msgr2=0x7f479006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f479006c8f0 0x7f479006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f47a414e0f0 tx=0x7f4798009040 comp rx=0 tx=0).stop 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 -- 172.21.15.40:0/2555278885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 msgr2=0x7f47a414d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 0x7f47a414d7c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e8a0 tx=0x7f479c00ed60 comp rx=0 tx=0).stop 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 -- 172.21.15.40:0/2555278885 shutdown_connections 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f479006c8f0 0x7f479006edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a4154c80 0x7f47a4156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a40c75f0 0x7f47a4156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 --2- 172.21.15.40:0/2555278885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a40c7030 0x7f47a414d7c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 -- 172.21.15.40:0/2555278885 >> 172.21.15.40:0/2555278885 conn(0x7f47a40727a0 msgr2=0x7f47a40cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 -- 172.21.15.40:0/2555278885 shutdown_connections 2024-04-25T11:38:59.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.555+0000 7f47ab564640 1 -- 172.21.15.40:0/2555278885 wait complete. 2024-04-25T11:38:59.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T11:38:59.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.668+0000 7f6fc95f0640 1 Processor -- start 2024-04-25T11:38:59.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.668+0000 7f6fc95f0640 1 -- start start 2024-04-25T11:38:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.668+0000 7f6fc95f0640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 0x7f6fc40cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.668+0000 7f6fc95f0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6fc40c7b70 0x7f6fc40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.668+0000 7f6fc95f0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fc40cc550 con 0x7f6fc40ccd00 2024-04-25T11:38:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.668+0000 7f6fc95f0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f6fc415ae60 con 0x7f6fc40c7b70 2024-04-25T11:38:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.668+0000 7f6fc95f0640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc415b140 con 0x7f6fc40cb240 2024-04-25T11:38:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc2ffd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 0x7f6fc40cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc27fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6fc40c7b70 0x7f6fc40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc27fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6fc40c7b70 0x7f6fc40cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.40:55462/0 (socket says 172.21.15.40:55462) 2024-04-25T11:38:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc37fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6fc40ccd00 0x7f6fc40cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:39712/0 (socket says 172.21.15.40:39712) 2024-04-25T11:38:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc27fc640 1 -- 172.21.15.40:0/3507226813 learned_addr learned my addr 172.21.15.40:0/3507226813 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc2ffd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 0x7f6fc40cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:37910/0 (socket says 172.21.15.40:37910) 2024-04-25T11:38:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc27fc640 1 -- 172.21.15.40:0/3507226813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 msgr2=0x7f6fc40cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/3507226813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 0x7f6fc40cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc27fc640 1 -- 172.21.15.40:0/3507226813 >> v1:172.21.15.62:6789/0 conn(0x7f6fc40ccd00 legacy=0x7f6fc40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.669+0000 7f6fc27fc640 1 -- 172.21.15.40:0/3507226813 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc415b420 con 0x7f6fc40c7b70 2024-04-25T11:38:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/3507226813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 0x7f6fc40cc010 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009850 tx=0x7f6fb802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db26202fdb814f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc1ffb640 1 -- 172.21.15.40:0/3507226813 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8004030 con 0x7f6fc40c7b70 2024-04-25T11:38:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc1ffb640 1 -- 172.21.15.40:0/3507226813 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb80041d0 con 0x7f6fc40c7b70 2024-04-25T11:38:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc1ffb640 1 -- 172.21.15.40:0/3507226813 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8003610 con 0x7f6fc40c7b70 2024-04-25T11:38:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc95f0640 1 -- 172.21.15.40:0/3507226813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 msgr2=0x7f6fc40cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc95f0640 1 --2- 172.21.15.40:0/3507226813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 0x7f6fc40cc010 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009850 tx=0x7f6fb802a230 comp rx=0 tx=0).stop 2024-04-25T11:38:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc95f0640 1 -- 172.21.15.40:0/3507226813 shutdown_connections 2024-04-25T11:38:59.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc95f0640 1 --2- 172.21.15.40:0/3507226813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 0x7f6fc40cc010 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc95f0640 1 --2- 172.21.15.40:0/3507226813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 0x7f6fc40cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc95f0640 1 -- 172.21.15.40:0/3507226813 >> 172.21.15.40:0/3507226813 conn(0x7f6fc406c3d0 msgr2=0x7f6fc406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc95f0640 1 -- 172.21.15.40:0/3507226813 shutdown_connections 2024-04-25T11:38:59.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.670+0000 7f6fc95f0640 1 -- 172.21.15.40:0/3507226813 wait complete. 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.672+0000 7f6fc95f0640 1 Processor -- start 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.672+0000 7f6fc95f0640 1 -- start start 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.672+0000 7f6fc95f0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 0x7f6fc4150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.672+0000 7f6fc95f0640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 0x7f6fc414e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.672+0000 7f6fc95f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc40ccd00 0x7f6fc414ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.672+0000 7f6fc95f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc41552b0 con 0x7f6fc40ccd00 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.672+0000 7f6fc95f0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc415b9e0 con 0x7f6fc40c7b70 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.672+0000 7f6fc95f0640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc40be4f0 con 0x7f6fc40cb240 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.672+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc40ccd00 0x7f6fc414ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc40ccd00 0x7f6fc414ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34852/0 (socket says 172.21.15.40:34852) 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 0x7f6fc4150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc37fe640 1 -- 172.21.15.40:0/4205287085 learned_addr learned my addr 172.21.15.40:0/4205287085 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 0x7f6fc414e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc37fe640 1 -- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 msgr2=0x7f6fc414e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc37fe640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 0x7f6fc414e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc37fe640 1 -- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 msgr2=0x7f6fc4150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc37fe640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 0x7f6fc4150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc37fe640 1 -- 172.21.15.40:0/4205287085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc40bb530 con 0x7f6fc40ccd00 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 0x7f6fc4150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc37fe640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc40ccd00 0x7f6fc414ed20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00c3c0 tx=0x7f6fbc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fab7fe640 1 -- 172.21.15.40:0/4205287085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0074e0 con 0x7f6fc40ccd00 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fab7fe640 1 -- 172.21.15.40:0/4205287085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fbc007680 con 0x7f6fc40ccd00 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc95f0640 1 -- 172.21.15.40:0/4205287085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc40bb7f0 con 0x7f6fc40ccd00 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fab7fe640 1 -- 172.21.15.40:0/4205287085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0049e0 con 0x7f6fc40ccd00 2024-04-25T11:38:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.673+0000 7f6fc95f0640 1 -- 172.21.15.40:0/4205287085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc40bd500 con 0x7f6fc40ccd00 2024-04-25T11:38:59.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.675+0000 7f6fab7fe640 1 -- 172.21.15.40:0/4205287085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fbc01a020 con 0x7f6fc40ccd00 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.675+0000 7f6fab7fe640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6fb006c920 0x7f6fb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.675+0000 7f6fab7fe640 1 -- 172.21.15.40:0/4205287085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6fbc08b170 con 0x7f6fc40ccd00 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.675+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6fb006c920 0x7f6fb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.676+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6fb006c920 0x7f6fb006ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400e740 tx=0x7f6fb400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.676+0000 7f6fcc3de740 1 Processor -- start 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.676+0000 7f6fcc3de740 1 -- start start 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566117a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d2fc0 0x5615660d6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 0x5615661218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56156603cc50 con 0x5615660d3690 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561565fd9830 con 0x5615660d3a70 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561566038c10 con 0x5615660d2fc0 2024-04-25T11:38:59.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566117a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566117a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34862/0 (socket says 172.21.15.40:34862) 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc2ffd640 1 -- 172.21.15.40:0/575527497 learned_addr learned my addr 172.21.15.40:0/575527497 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc37fe640 1 --2- 172.21.15.40:0/575527497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 0x5615661218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/575527497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d2fc0 0x5615660d6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc2ffd640 1 -- 172.21.15.40:0/575527497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d2fc0 msgr2=0x5615660d6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/575527497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d2fc0 0x5615660d6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc2ffd640 1 -- 172.21.15.40:0/575527497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 msgr2=0x5615661218a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/575527497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 0x5615661218a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc2ffd640 1 -- 172.21.15.40:0/575527497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561565d4f2d0 con 0x5615660d3690 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/575527497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566117a90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4007ae0 tx=0x7f6fb4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a774ec5b3ce8139 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.677+0000 7f6fa8ff9640 1 -- 172.21.15.40:0/575527497 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4015410 con 0x5615660d3690 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fa8ff9640 1 -- 172.21.15.40:0/575527497 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb401b070 con 0x5615660d3690 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fa8ff9640 1 -- 172.21.15.40:0/575527497 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40157d0 con 0x5615660d3690 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fcc3de740 1 -- 172.21.15.40:0/575527497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 msgr2=0x561566117a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/575527497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566117a90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4007ae0 tx=0x7f6fb4014000 comp rx=0 tx=0).stop 2024-04-25T11:38:59.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fcc3de740 1 -- 172.21.15.40:0/575527497 shutdown_connections 2024-04-25T11:38:59.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/575527497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 0x5615661218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/575527497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d2fc0 0x5615660d6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/575527497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566117a90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fcc3de740 1 -- 172.21.15.40:0/575527497 >> 172.21.15.40:0/575527497 conn(0x5615660d4c80 msgr2=0x5615660d7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fcc3de740 1 -- 172.21.15.40:0/575527497 shutdown_connections 2024-04-25T11:38:59.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.678+0000 7f6fcc3de740 1 -- 172.21.15.40:0/575527497 wait complete. 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.679+0000 7f6fcc3de740 1 -- start start 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.679+0000 7f6fcc3de740 1 Processor -- start 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 0x5615660f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d3a70 0x5615660fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660f6040 0x56156612cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x561566118700 con 0x5615660f6040 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561565fd7470 con 0x5615660d3690 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5615660d60e0 con 0x5615660d3a70 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660f6040 0x56156612cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fcc3de740 10 client.0 _ll_register_callbacks cb 0x561565ffcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660f6040 0x56156612cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34870/0 (socket says 172.21.15.40:34870) 2024-04-25T11:38:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc37fe640 1 -- 172.21.15.40:0/1054688236 learned_addr learned my addr 172.21.15.40:0/1054688236 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 0x5615660f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d3a70 0x5615660fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc37fe640 1 -- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d3a70 msgr2=0x5615660fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc37fe640 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d3a70 0x5615660fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc37fe640 1 -- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 msgr2=0x5615660f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc37fe640 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 0x5615660f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc37fe640 1 -- 172.21.15.40:0/1054688236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561565e196b0 con 0x5615660f6040 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fc37fe640 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660f6040 0x56156612cc10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc001eb0 tx=0x7f6fbc05d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c3ffe6de9bdbea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1054688236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc057310 con 0x5615660f6040 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1054688236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6fbc0593c0 con 0x5615660f6040 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.680+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1054688236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb401b070 con 0x5615660f6040 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1054688236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f6fbc0615f0 con 0x5615660f6040 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fcc3de740 10 client.4770 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1054688236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x561566100500 con 0x5615660f6040 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1054688236 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb401b070 con 0x5615660f6040 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1054688236 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f6fbc0711d0 con 0x5615660f6040 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fcc3de740 10 client.4770 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fcc3de740 10 client.4770 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fcc3de740 20 client.4770 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] conn(0x561566136540 0x561566138930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fcc3de740 4 client.4770 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1054688236 --> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56156604a670 con 0x561566136540 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] conn(0x561566136540 0x561566138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] conn(0x561566136540 0x561566138930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8414fdecec03bef server_cookie=51b43affd84b6868 in_seq=0 out_seq=0 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.681+0000 7f6fa1ffb640 10 client.4770 ms_handle_connect on v2:172.21.15.62:6837/2121129816 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1054688236 <== mds.0 v2:172.21.15.62:6837/2121129816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56156604a670 con 0x561566136540 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fa1ffb640 10 client.4770 handle_command_reply: tid=0 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 client.4770 shutdown 2024-04-25T11:38:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1054688236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660f6040 msgr2=0x56156612cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660f6040 0x56156612cc10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc001eb0 tx=0x7f6fbc05d310 comp rx=0 tx=0).stop 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1054688236 shutdown_connections 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] conn(0x561566136540 0x561566138930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660f6040 0x56156612cc10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660d3a70 0x5615660fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fa1ffb640 0 client.4770 ms_handle_reset on v2:172.21.15.62:6837/2121129816 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1054688236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 0x5615660f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1054688236 >> 172.21.15.40:0/1054688236 conn(0x5615660d2fc0 msgr2=0x5615660d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1054688236 shutdown_connections 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1054688236 wait complete. 2024-04-25T11:38:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.682+0000 7f6fcc3de740 20 client.4770 trim_cache size 0 max 16384 2024-04-25T11:38:59.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fcc3de740 1 Processor -- start 2024-04-25T11:38:59.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fcc3de740 1 -- start start 2024-04-25T11:38:59.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 0x5615660f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3a70 0x56156613d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660f6040 0x56156613dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc41fd2b0 con 0x5615660d3a70 2024-04-25T11:38:59.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc40752d0 con 0x5615660d3690 2024-04-25T11:38:59.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561566100710 con 0x5615660f6040 2024-04-25T11:38:59.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fc27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3a70 0x56156613d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fc27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3a70 0x56156613d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34884/0 (socket says 172.21.15.40:34884) 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.701+0000 7f6fc27fc640 1 -- 172.21.15.40:0/94513783 learned_addr learned my addr 172.21.15.40:0/94513783 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/94513783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 0x5615660f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fc27fc640 1 -- 172.21.15.40:0/94513783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660f6040 msgr2=0x56156613dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/94513783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660f6040 0x56156613dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fc27fc640 1 -- 172.21.15.40:0/94513783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 msgr2=0x5615660f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/94513783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 0x5615660f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fc27fc640 1 -- 172.21.15.40:0/94513783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615661175e0 con 0x5615660d3a70 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/94513783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3a70 0x56156613d960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8000c00 tx=0x7f6fb802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9604a03e3b58b2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fa3fff640 1 -- 172.21.15.40:0/94513783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80042a0 con 0x5615660d3a70 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fa3fff640 1 -- 172.21.15.40:0/94513783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb8004440 con 0x5615660d3a70 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fa3fff640 1 -- 172.21.15.40:0/94513783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8005990 con 0x5615660d3a70 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fcc3de740 1 -- 172.21.15.40:0/94513783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3a70 msgr2=0x56156613d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/94513783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3a70 0x56156613d960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8000c00 tx=0x7f6fb802c000 comp rx=0 tx=0).stop 2024-04-25T11:38:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fcc3de740 1 -- 172.21.15.40:0/94513783 shutdown_connections 2024-04-25T11:38:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/94513783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660f6040 0x56156613dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/94513783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3a70 0x56156613d960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/94513783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3690 0x5615660f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fcc3de740 1 -- 172.21.15.40:0/94513783 >> 172.21.15.40:0/94513783 conn(0x5615660d2fc0 msgr2=0x561566116960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fcc3de740 1 -- 172.21.15.40:0/94513783 shutdown_connections 2024-04-25T11:38:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.702+0000 7f6fcc3de740 1 -- 172.21.15.40:0/94513783 wait complete. 2024-04-25T11:38:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.703+0000 7f6fcc3de740 1 -- start start 2024-04-25T11:38:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.703+0000 7f6fcc3de740 1 Processor -- start 2024-04-25T11:38:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566118a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 0x56156612c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fcc3de740 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660f6040 0x56156618d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56156612d930 con 0x5615660d3690 2024-04-25T11:38:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5615660fdfd0 con 0x5615660d3a70 2024-04-25T11:38:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fcc3de740 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5615660f26d0 con 0x5615660f6040 2024-04-25T11:38:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fcc3de740 10 client.0 _ll_register_callbacks cb 0x7f6fbc055240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566118a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566118a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34892/0 (socket says 172.21.15.40:34892) 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc2ffd640 1 -- 172.21.15.40:0/1503198036 learned_addr learned my addr 172.21.15.40:0/1503198036 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 0x56156612c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc37fe640 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660f6040 0x56156618d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc2ffd640 1 -- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660f6040 msgr2=0x56156618d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660f6040 0x56156618d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc2ffd640 1 -- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 msgr2=0x56156612c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 0x56156612c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc2ffd640 1 -- 172.21.15.40:0/1503198036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615660f2500 con 0x5615660d3690 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 0x56156612c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fc2ffd640 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566118a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4007890 tx=0x7f6fb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61306d61e02e7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1503198036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb403d060 con 0x5615660d3690 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1503198036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6fb4038600 con 0x5615660d3690 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.704+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1503198036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb8004440 con 0x5615660d3690 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1503198036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f6fb4043050 con 0x5615660d3690 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fcc3de740 10 client.4776 fetch_fsmap learned FSMap version 17 2024-04-25T11:38:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1503198036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x561566101950 con 0x5615660d3690 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1503198036 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb8004440 con 0x5615660d3690 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1503198036 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f6fb4046020 con 0x5615660d3690 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fcc3de740 10 client.4776 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fcc3de740 10 client.4776 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fcc3de740 20 client.4776 populate_metadata read hostname 'smithi040' 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] conn(0x561566197700 0x5615661a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fcc3de740 4 client.4776 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1503198036 --> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] -- command(tid 0: {"prefix": "status"}) -- 0x5615661184b0 con 0x561566197700 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] conn(0x561566197700 0x5615661a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fa1ffb640 10 client.4776 ms_handle_connect on v2:172.21.15.62:6837/2121129816 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.705+0000 7f6fc27fc640 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] conn(0x561566197700 0x5615661a02a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e7a733eb85f5099 server_cookie=d29de0782b83b337 in_seq=0 out_seq=0 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fa1ffb640 1 -- 172.21.15.40:0/1503198036 <== mds.0 v2:172.21.15.62:6837/2121129816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5615661184b0 con 0x561566197700 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fa1ffb640 10 client.4776 handle_command_reply: tid=0 2024-04-25T11:38:59.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 client.4776 shutdown 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1503198036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 msgr2=0x561566118a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566118a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4007890 tx=0x7f6fb4004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1503198036 shutdown_connections 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:6837/2121129816,v1:172.21.15.62:6839/2121129816] conn(0x561566197700 0x5615661a02a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fa1ffb640 0 client.4776 ms_handle_reset on v2:172.21.15.62:6837/2121129816 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5615660f6040 0x56156618d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5615660d3a70 0x56156612c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 --2- 172.21.15.40:0/1503198036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615660d3690 0x561566118a40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1503198036 >> 172.21.15.40:0/1503198036 conn(0x5615660d2fc0 msgr2=0x5615660f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1503198036 shutdown_connections 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 1 -- 172.21.15.40:0/1503198036 wait complete. 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fcc3de740 20 client.4776 trim_cache size 0 max 16384 2024-04-25T11:38:59.699 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "6d14dde7-764d-42d0-be37-c790a704858d", 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4300, 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 34.039076666, 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 17, 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 20, 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 40.432471096999997 2024-04-25T11:38:59.700 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:38:59.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 -- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6fb006c920 msgr2=0x7f6fb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6fb006c920 0x7f6fb006ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400e740 tx=0x7f6fb400b040 comp rx=0 tx=0).stop 2024-04-25T11:38:59.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 -- 172.21.15.40:0/4205287085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc40ccd00 msgr2=0x7f6fc414ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc40ccd00 0x7f6fc414ed20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00c3c0 tx=0x7f6fbc00c880 comp rx=0 tx=0).stop 2024-04-25T11:38:59.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 -- 172.21.15.40:0/4205287085 shutdown_connections 2024-04-25T11:38:59.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6fb006c920 0x7f6fb006ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fc40ccd00 0x7f6fc414ed20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fc40cb240 0x7f6fc414e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 --2- 172.21.15.40:0/4205287085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fc40c7b70 0x7f6fc4150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.706+0000 7f6fc95f0640 1 -- 172.21.15.40:0/4205287085 >> 172.21.15.40:0/4205287085 conn(0x7f6fc406c3d0 msgr2=0x7f6fc40c33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.707+0000 7f6fc95f0640 1 -- 172.21.15.40:0/4205287085 shutdown_connections 2024-04-25T11:38:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.707+0000 7f6fc95f0640 1 -- 172.21.15.40:0/4205287085 wait complete. 2024-04-25T11:38:59.705 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T11:38:59.718 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T11:38:59.719 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T11:38:59.719 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T11:38:59.719 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T11:38:59.743 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T11:38:59.743 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-25T11:38:59.744 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T11:38:59.744 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:38:59.744 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:38:59.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:59.744 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-25T11:38:59.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:59.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-25T11:38:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-04-25T11:38:59.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:38:59.828 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T11:38:59.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:38:59.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.981+0000 7f86dc6e1640 1 Processor -- start 2024-04-25T11:38:59.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.981+0000 7f86dc6e1640 1 -- start start 2024-04-25T11:38:59.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.982+0000 7f86dc6e1640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40c7b70 0x7f86d40cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.982+0000 7f86dc6e1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d415ae60 con 0x7f86d40ccd00 2024-04-25T11:38:59.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.982+0000 7f86dc6e1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d415b140 con 0x7f86d40cb240 2024-04-25T11:38:59.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.982+0000 7f86dc6e1640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f86d415b420 con 0x7f86d40c7b70 2024-04-25T11:38:59.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.982+0000 7f86dac7e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f86d40ccd00 0x7f86d40cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37652/0 (socket says 172.21.15.40:37652) 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.982+0000 7f86dac7e640 1 -- 172.21.15.40:0/714537491 learned_addr learned my addr 172.21.15.40:0/714537491 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.983+0000 7f86d9c7c640 1 --2- 172.21.15.40:0/714537491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40c7b70 0x7f86d40cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.983+0000 7f86d947b640 1 -- 172.21.15.40:0/714537491 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596904644 0 0) 0x7f86d415ae60 con 0x7f86d40ccd00 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.983+0000 7f86d947b640 1 -- 172.21.15.40:0/714537491 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86bc003520 con 0x7f86d40ccd00 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.983+0000 7f86d947b640 1 -- 172.21.15.40:0/714537491 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 736192941 0 0) 0x7f86d415b140 con 0x7f86d40cb240 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.983+0000 7f86d947b640 1 -- 172.21.15.40:0/714537491 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86d415ae60 con 0x7f86d40cb240 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.983+0000 7f86d9c7c640 1 -- 172.21.15.40:0/714537491 >> v1:172.21.15.62:6789/0 conn(0x7f86d40cb240 legacy=0x7f86d40cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.983+0000 7f86d9c7c640 1 -- 172.21.15.40:0/714537491 >> v1:172.21.15.40:6789/0 conn(0x7f86d40ccd00 legacy=0x7f86d40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.983+0000 7f86d947b640 1 -- 172.21.15.40:0/714537491 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3713139431 0 0) 0x7f86bc003520 con 0x7f86d40ccd00 2024-04-25T11:38:59.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.983+0000 7f86d9c7c640 1 -- 172.21.15.40:0/714537491 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d415b700 con 0x7f86d40c7b70 2024-04-25T11:38:59.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.984+0000 7f86d9c7c640 1 --2- 172.21.15.40:0/714537491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40c7b70 0x7f86d40cb620 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f86c80097c0 tx=0x7f86c802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec903c9c30b91de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.984+0000 7f86d947b640 1 -- 172.21.15.40:0/714537491 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8037070 con 0x7f86d40c7b70 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.984+0000 7f86d947b640 1 -- 172.21.15.40:0/714537491 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c802ac10 con 0x7f86d40c7b70 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.984+0000 7f86d947b640 1 -- 172.21.15.40:0/714537491 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c40019f0 con 0x7f86d40c7b70 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.984+0000 7f86dc6e1640 1 -- 172.21.15.40:0/714537491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40c7b70 msgr2=0x7f86d40cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.984+0000 7f86dc6e1640 1 --2- 172.21.15.40:0/714537491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40c7b70 0x7f86d40cb620 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f86c80097c0 tx=0x7f86c802a180 comp rx=0 tx=0).stop 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.984+0000 7f86dc6e1640 1 -- 172.21.15.40:0/714537491 shutdown_connections 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.984+0000 7f86dc6e1640 1 --2- 172.21.15.40:0/714537491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40c7b70 0x7f86d40cb620 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.984+0000 7f86dc6e1640 1 -- 172.21.15.40:0/714537491 >> 172.21.15.40:0/714537491 conn(0x7f86d406c3d0 msgr2=0x7f86d406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.985+0000 7f86dc6e1640 1 -- 172.21.15.40:0/714537491 shutdown_connections 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.985+0000 7f86dc6e1640 1 -- 172.21.15.40:0/714537491 wait complete. 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dc6e1640 1 Processor -- start 2024-04-25T11:38:59.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dc6e1640 1 -- start start 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dc6e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d40c7b70 0x7f86d41f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dc6e1640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40cb240 0x7f86d41586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dc6e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d40ccd00 0x7f86d40c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dc6e1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41552b0 con 0x7f86d40ccd00 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dc6e1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f86d40b6290 con 0x7f86d40c7b70 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dc6e1640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f86d40c5f50 con 0x7f86d40cb240 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dac7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d40ccd00 0x7f86d40c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.986+0000 7f86dac7e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d40ccd00 0x7f86d40c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34906/0 (socket says 172.21.15.40:34906) 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86dac7e640 1 -- 172.21.15.40:0/1660876362 learned_addr learned my addr 172.21.15.40:0/1660876362 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86da47d640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d40c7b70 0x7f86d41f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86d9c7c640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40cb240 0x7f86d41586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86dac7e640 1 -- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40cb240 msgr2=0x7f86d41586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86dac7e640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40cb240 0x7f86d41586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86dac7e640 1 -- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d40c7b70 msgr2=0x7f86d41f07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86dac7e640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d40c7b70 0x7f86d41f07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86dac7e640 1 -- 172.21.15.40:0/1660876362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d40c50d0 con 0x7f86d40ccd00 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86d9c7c640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40cb240 0x7f86d41586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86da47d640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d40c7b70 0x7f86d41f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86dac7e640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d40ccd00 0x7f86d40c63e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86c400cbe0 tx=0x7f86c400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86d2ffd640 1 -- 172.21.15.40:0/1660876362 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c400ad40 con 0x7f86d40ccd00 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86d2ffd640 1 -- 172.21.15.40:0/1660876362 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c4012400 con 0x7f86d40ccd00 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d40c5340 con 0x7f86d40ccd00 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86d2ffd640 1 -- 172.21.15.40:0/1660876362 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c40127d0 con 0x7f86d40ccd00 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.987+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d40c5900 con 0x7f86d40ccd00 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.988+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d40692e0 con 0x7f86d40ccd00 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.988+0000 7f86d2ffd640 1 -- 172.21.15.40:0/1660876362 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86c4012970 con 0x7f86d40ccd00 2024-04-25T11:38:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.989+0000 7f86d2ffd640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.989+0000 7f86d2ffd640 1 -- 172.21.15.40:0/1660876362 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f86c408c2f0 con 0x7f86d40ccd00 2024-04-25T11:38:59.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.989+0000 7f86da47d640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.990+0000 7f86da47d640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f86cc001710 tx=0x7f86cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:38:59.993+0000 7f86d2ffd640 1 -- 172.21.15.40:0/1660876362 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86d40692e0 con 0x7f86d40ccd00 2024-04-25T11:39:00.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.218+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f86d4068bb0 con 0x7f86d40ccd00 2024-04-25T11:39:00.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.218+0000 7f86d2ffd640 1 -- 172.21.15.40:0/1660876362 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f86c40182b0 con 0x7f86d40ccd00 2024-04-25T11:39:00.206 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:39:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.220+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f86c006c8f0 msgr2=0x7f86c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.220+0000 7f86dc6e1640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f86cc001710 tx=0x7f86cc009040 comp rx=0 tx=0).stop 2024-04-25T11:39:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.220+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d40ccd00 msgr2=0x7f86d40c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.220+0000 7f86dc6e1640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d40ccd00 0x7f86d40c63e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86c400cbe0 tx=0x7f86c400a430 comp rx=0 tx=0).stop 2024-04-25T11:39:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.220+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 shutdown_connections 2024-04-25T11:39:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.220+0000 7f86dc6e1640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.220+0000 7f86dc6e1640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d40ccd00 0x7f86d40c63e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.220+0000 7f86dc6e1640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86d40cb240 0x7f86d41586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.220+0000 7f86dc6e1640 1 --2- 172.21.15.40:0/1660876362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d40c7b70 0x7f86d41f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.221+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 >> 172.21.15.40:0/1660876362 conn(0x7f86d406c3d0 msgr2=0x7f86d414e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.221+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 shutdown_connections 2024-04-25T11:39:00.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00.221+0000 7f86dc6e1640 1 -- 172.21.15.40:0/1660876362 wait complete. 2024-04-25T11:39:00.218 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:39:00.218 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:39:00.218 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-25T11:39:00.218 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:39:00.219 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:00.219 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:39:00.219 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:39:00.219 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:39:00.219 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:39:00.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:00.219 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-25T11:39:00.234 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:39:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42711sec preferred_lft 42711sec 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:39:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-25T11:39:00.237 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:39:00.237 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:39:00.237 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-25T11:39:00.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:39:00.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-04-25T11:39:00.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-04-25T11:39:00.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:39:00.237 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-25T11:39:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:39:00.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:39:00.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:00.424 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:39:00.449 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-25T11:39:00.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:00.451 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-25T11:39:00.520 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-25T11:39:00.520 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:39:00.520 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:39:00.520 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:39:00.520 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:39:00.520 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:39:00.521 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-25T11:39:00.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:39:00.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:39:00.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:39:00.521 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-25T11:39:00.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:39:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:39:00.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:00.799 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-25T11:39:00.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:00.815 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-04-25T11:39:00.870 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:39:00.871 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-25T11:39:00.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:39:00.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T11:39:00.871 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-25T11:39:00.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:39:00.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:39:00.992 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T11:39:00.992 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:39:00.992 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-25T11:39:00.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T11:39:00.993 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:39:00.993 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T11:39:00.993 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T11:39:00.993 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T11:39:00.993 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-25T11:39:01.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:39:01.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:39:01.123 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:39:01.123 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-25T11:39:01.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-25T11:39:01.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:39:01.123 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-25T11:39:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:39:01.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:39:01.203 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T11:39:01.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:39:01.203 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:01.218 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:39:01.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:39:01.219 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:01.274 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-25T11:39:01.340 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:39:01.396 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-25T11:39:01.397 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/selinux 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-25T11:39:01.398 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:39:01.399 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:39:01.399 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:39:01.399 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:39:01.399 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-25T11:39:01.399 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:39:01.399 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:39:01.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:01.399 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-25T11:39:01.451 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T11:39:01.452 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T11:39:01.494 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-25T11:39:01.519 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:39:01.575 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-25T11:39:01.575 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/selinux 2024-04-25T11:39:01.576 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:39:01.577 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:39:01.578 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:39:01.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:01.579 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-25T11:39:01.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:2024-04-25T11:39:01.597+0000 7efdfcd1e480 -1 init, newargv = 0x55b979aca880 newargc=15 2024-04-25T11:39:01.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[42246]: starting ceph client 2024-04-25T11:39:01.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[42246]: starting fuse 2024-04-25T11:39:01.601 INFO:teuthology.orchestra.run.smithi040.stdout:43 2024-04-25T11:39:01.601 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T11:39:01.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:39:01.601 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2024-04-25T11:39:01.601 DEBUG:teuthology.orchestra.run.smithi040:> import re 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2024-04-25T11:39:01.602 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> return f 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.0")) 2024-04-25T11:39:01.603 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-25T11:39:01.720 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.0.42246.asok 2024-04-25T11:39:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:39:01.731 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42246.asok 2024-04-25T11:39:01.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:01.731 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42246.asok status 2024-04-25T11:39:01.867 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "d69f13d827952bb07bb9b7973bca14f97d764d6f", 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "0", 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "42246", 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4788, 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4788 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3068897779 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-25T11:39:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3068897779 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4788 192.168.0.1:0/3068897779", 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.1:0/3068897779", 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 17, 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 20, 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2024-04-25T11:39:01.870 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:39:01.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:01.876 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2024-04-25T11:39:01.876 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:01.892 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2024-04-25T11:39:01.892 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:01.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:01.893 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:01.960 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T11:39:01.973 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks/quiescer.py", line 386, in task cluster_name = ctx.config['cluster'] KeyError: 'cluster' 2024-04-25T11:39:02.219 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b453a7003ff14de5865b8c074fd41511 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks/quiescer.py", line 386, in task cluster_name = ctx.config['cluster'] KeyError: 'cluster' 2024-04-25T11:39:02.221 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T11:39:02.230 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T11:39:02.238 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T11:39:02.238 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:39:02.238 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:02.258 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:39:02.258 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:02.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:39:02.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.468+0000 7f0f3edfe640 1 Processor -- start 2024-04-25T11:39:02.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.468+0000 7f0f3edfe640 1 -- start start 2024-04-25T11:39:02.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3edfe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f38159840 0x7f0f380c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3edfe640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f381565e0 con 0x7f0f38156200 2024-04-25T11:39:02.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3edfe640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f380c8fc0 con 0x7f0f380c85c0 2024-04-25T11:39:02.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3edfe640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0f380c33e0 con 0x7f0f38159840 2024-04-25T11:39:02.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3d39b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f38159840 0x7f0f380c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3cb9a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0f38156200 0x7f0f380c7a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37662/0 (socket says 172.21.15.40:37662) 2024-04-25T11:39:02.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3d39b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f38159840 0x7f0f380c2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34908/0 (socket says 172.21.15.40:34908) 2024-04-25T11:39:02.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3cb9a640 1 -- 172.21.15.40:0/486210464 learned_addr learned my addr 172.21.15.40:0/486210464 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:02.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3d39b640 1 -- 172.21.15.40:0/486210464 >> v1:172.21.15.62:6789/0 conn(0x7f0f380c85c0 legacy=0x7f0f38159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3d39b640 1 -- 172.21.15.40:0/486210464 >> v1:172.21.15.40:6789/0 conn(0x7f0f38156200 legacy=0x7f0f380c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f377fe640 1 -- 172.21.15.40:0/486210464 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4208173331 0 0) 0x7f0f381565e0 con 0x7f0f38156200 2024-04-25T11:39:02.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.469+0000 7f0f3d39b640 1 -- 172.21.15.40:0/486210464 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f380c92a0 con 0x7f0f38159840 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.470+0000 7f0f3d39b640 1 --2- 172.21.15.40:0/486210464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38159840 0x7f0f380c2ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f280097a0 tx=0x7f0f2802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bc31c96de0d497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.470+0000 7f0f377fe640 1 -- 172.21.15.40:0/486210464 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28037070 con 0x7f0f38159840 2024-04-25T11:39:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.470+0000 7f0f377fe640 1 -- 172.21.15.40:0/486210464 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f28032640 con 0x7f0f38159840 2024-04-25T11:39:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.470+0000 7f0f377fe640 1 -- 172.21.15.40:0/486210464 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28032a10 con 0x7f0f38159840 2024-04-25T11:39:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.471+0000 7f0f3edfe640 1 -- 172.21.15.40:0/486210464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38159840 msgr2=0x7f0f380c2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.471+0000 7f0f3edfe640 1 --2- 172.21.15.40:0/486210464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38159840 0x7f0f380c2ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f280097a0 tx=0x7f0f2802a2c0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.471+0000 7f0f3edfe640 1 -- 172.21.15.40:0/486210464 shutdown_connections 2024-04-25T11:39:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.471+0000 7f0f3edfe640 1 --2- 172.21.15.40:0/486210464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38159840 0x7f0f380c2ea0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.471+0000 7f0f3edfe640 1 -- 172.21.15.40:0/486210464 >> 172.21.15.40:0/486210464 conn(0x7f0f3806b970 msgr2=0x7f0f3806bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:02.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.471+0000 7f0f3edfe640 1 -- 172.21.15.40:0/486210464 shutdown_connections 2024-04-25T11:39:02.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.471+0000 7f0f3edfe640 1 -- 172.21.15.40:0/486210464 wait complete. 2024-04-25T11:39:02.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.472+0000 7f0f3edfe640 1 Processor -- start 2024-04-25T11:39:02.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.472+0000 7f0f3edfe640 1 -- start start 2024-04-25T11:39:02.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f3edfe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f380c85c0 0x7f0f380731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f3edfe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38156200 0x7f0f380cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f3edfe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f38159840 0x7f0f38071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f3edfe640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0f380c9860 con 0x7f0f38156200 2024-04-25T11:39:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f3edfe640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0f380c8d70 con 0x7f0f380c85c0 2024-04-25T11:39:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f3edfe640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0f380c8190 con 0x7f0f38159840 2024-04-25T11:39:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38156200 0x7f0f380cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38156200 0x7f0f380cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34924/0 (socket says 172.21.15.40:34924) 2024-04-25T11:39:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f37fff640 1 -- 172.21.15.40:0/3279830287 learned_addr learned my addr 172.21.15.40:0/3279830287 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f3d39b640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f38159840 0x7f0f38071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f3cb9a640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f380c85c0 0x7f0f380731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f37fff640 1 -- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f38159840 msgr2=0x7f0f38071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f37fff640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f38159840 0x7f0f38071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f37fff640 1 -- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f380c85c0 msgr2=0x7f0f380731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f37fff640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f380c85c0 0x7f0f380731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f37fff640 1 -- 172.21.15.40:0/3279830287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f381f1530 con 0x7f0f38156200 2024-04-25T11:39:02.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.473+0000 7f0f3d39b640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f38159840 0x7f0f38071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:02.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.474+0000 7f0f37fff640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38156200 0x7f0f380cb6a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00c450 tx=0x7f0f2c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.474+0000 7f0f357fa640 1 -- 172.21.15.40:0/3279830287 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0052d0 con 0x7f0f38156200 2024-04-25T11:39:02.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.474+0000 7f0f357fa640 1 -- 172.21.15.40:0/3279830287 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2c005470 con 0x7f0f38156200 2024-04-25T11:39:02.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.474+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f381f0770 con 0x7f0f38156200 2024-04-25T11:39:02.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.474+0000 7f0f357fa640 1 -- 172.21.15.40:0/3279830287 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c012870 con 0x7f0f38156200 2024-04-25T11:39:02.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.474+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f381f0d30 con 0x7f0f38156200 2024-04-25T11:39:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.475+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f38155980 con 0x7f0f38156200 2024-04-25T11:39:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.475+0000 7f0f357fa640 1 -- 172.21.15.40:0/3279830287 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f2c012a10 con 0x7f0f38156200 2024-04-25T11:39:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.475+0000 7f0f357fa640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f2406c8f0 0x7f0f2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.476+0000 7f0f357fa640 1 -- 172.21.15.40:0/3279830287 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0f2c08b6c0 con 0x7f0f38156200 2024-04-25T11:39:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.476+0000 7f0f3cb9a640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f2406c8f0 0x7f0f2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.476+0000 7f0f3cb9a640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f2406c8f0 0x7f0f2406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0f300012e0 tx=0x7f0f30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.480+0000 7f0f357fa640 1 -- 172.21.15.40:0/3279830287 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f38155980 con 0x7f0f38156200 2024-04-25T11:39:02.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.667+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f38068bb0 con 0x7f0f38156200 2024-04-25T11:39:02.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.669+0000 7f0f357fa640 1 -- 172.21.15.40:0/3279830287 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f0f2c059b80 con 0x7f0f38156200 2024-04-25T11:39:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:02.656 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:26.664101+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:39:02.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f2406c8f0 msgr2=0x7f0f2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f2406c8f0 0x7f0f2406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0f300012e0 tx=0x7f0f30009040 comp rx=0 tx=0).stop 2024-04-25T11:39:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38156200 msgr2=0x7f0f380cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38156200 0x7f0f380cb6a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00c450 tx=0x7f0f2c00c910 comp rx=0 tx=0).stop 2024-04-25T11:39:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 shutdown_connections 2024-04-25T11:39:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f0f2406c8f0 0x7f0f2406edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f38159840 0x7f0f38071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f38156200 0x7f0f380cb6a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 --2- 172.21.15.40:0/3279830287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f380c85c0 0x7f0f380731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 >> 172.21.15.40:0/3279830287 conn(0x7f0f3806b970 msgr2=0x7f0f380cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:02.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 shutdown_connections 2024-04-25T11:39:02.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.671+0000 7f0f3edfe640 1 -- 172.21.15.40:0/3279830287 wait complete. 2024-04-25T11:39:02.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:38:18.972986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:38:23.454183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:38:23.968333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:39:02.670 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:39:02.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.786+0000 7fd84a2a0640 1 Processor -- start 2024-04-25T11:39:02.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.786+0000 7fd84a2a0640 1 -- start start 2024-04-25T11:39:02.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd84a2a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8440c71e0 0x7fd84406c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd84a2a0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8440ca6d0 con 0x7fd8440c6e00 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd84a2a0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8440ca9a0 con 0x7fd844072b50 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd84a2a0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd844073890 con 0x7fd8440c71e0 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd8437fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8440c71e0 0x7fd84406c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd843fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd8440c6e00 0x7fd84415c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52672/0 (socket says 172.21.15.40:52672) 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd843fff640 1 -- 172.21.15.40:0/1747639121 learned_addr learned my addr 172.21.15.40:0/1747639121 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd8437fe640 1 -- 172.21.15.40:0/1747639121 >> v1:172.21.15.62:6789/0 conn(0x7fd844072b50 legacy=0x7fd844152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd8437fe640 1 -- 172.21.15.40:0/1747639121 >> v1:172.21.15.40:6789/0 conn(0x7fd8440c6e00 legacy=0x7fd84415c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd8437fe640 1 -- 172.21.15.40:0/1747639121 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8440cac80 con 0x7fd8440c71e0 2024-04-25T11:39:02.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.787+0000 7fd842ffd640 1 -- 172.21.15.40:0/1747639121 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4054211592 0 0) 0x7fd8440ca6d0 con 0x7fd8440c6e00 2024-04-25T11:39:02.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.788+0000 7fd8437fe640 1 --2- 172.21.15.40:0/1747639121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c71e0 0x7fd84406c450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8340097a0 tx=0x7fd83402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bfedcebe2cba69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.788+0000 7fd842ffd640 1 -- 172.21.15.40:0/1747639121 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834037070 con 0x7fd8440c71e0 2024-04-25T11:39:02.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.788+0000 7fd842ffd640 1 -- 172.21.15.40:0/1747639121 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd834032640 con 0x7fd8440c71e0 2024-04-25T11:39:02.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.788+0000 7fd842ffd640 1 -- 172.21.15.40:0/1747639121 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834032a10 con 0x7fd8440c71e0 2024-04-25T11:39:02.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.788+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1747639121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c71e0 msgr2=0x7fd84406c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.788+0000 7fd84a2a0640 1 --2- 172.21.15.40:0/1747639121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c71e0 0x7fd84406c450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8340097a0 tx=0x7fd83402a2c0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.789+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1747639121 shutdown_connections 2024-04-25T11:39:02.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.789+0000 7fd84a2a0640 1 --2- 172.21.15.40:0/1747639121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c71e0 0x7fd84406c450 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.789+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1747639121 >> 172.21.15.40:0/1747639121 conn(0x7fd84406ca80 msgr2=0x7fd84406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:02.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.789+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1747639121 shutdown_connections 2024-04-25T11:39:02.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.789+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1747639121 wait complete. 2024-04-25T11:39:02.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.790+0000 7fd84a2a0640 1 Processor -- start 2024-04-25T11:39:02.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.790+0000 7fd84a2a0640 1 -- start start 2024-04-25T11:39:02.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd84a2a0640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd844072b50 0x7fd8440c1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd84a2a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c6e00 0x7fd8440c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd84a2a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8440c71e0 0x7fd84415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd84a2a0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8440c9f10 con 0x7fd8440c6e00 2024-04-25T11:39:02.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd84a2a0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd844073520 con 0x7fd8440c71e0 2024-04-25T11:39:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd84a2a0640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd84415c8f0 con 0x7fd844072b50 2024-04-25T11:39:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c6e00 0x7fd8440c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c6e00 0x7fd8440c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45456/0 (socket says 172.21.15.40:45456) 2024-04-25T11:39:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd8437fe640 1 -- 172.21.15.40:0/1224533245 learned_addr learned my addr 172.21.15.40:0/1224533245 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd843fff640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd844072b50 0x7fd8440c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd84883d640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8440c71e0 0x7fd84415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd8437fe640 1 -- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd844072b50 msgr2=0x7fd8440c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd8437fe640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd844072b50 0x7fd8440c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd8437fe640 1 -- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8440c71e0 msgr2=0x7fd84415b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd8437fe640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8440c71e0 0x7fd84415b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd8437fe640 1 -- 172.21.15.40:0/1224533245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8440c1ae0 con 0x7fd8440c6e00 2024-04-25T11:39:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd8437fe640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c6e00 0x7fd8440c3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd8340097a0 tx=0x7fd8340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd840ff9640 1 -- 172.21.15.40:0/1224533245 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834037050 con 0x7fd8440c6e00 2024-04-25T11:39:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd840ff9640 1 -- 172.21.15.40:0/1224533245 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd834038040 con 0x7fd8440c6e00 2024-04-25T11:39:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8441f1ee0 con 0x7fd8440c6e00 2024-04-25T11:39:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd840ff9640 1 -- 172.21.15.40:0/1224533245 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834004490 con 0x7fd8440c6e00 2024-04-25T11:39:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.791+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8441f24a0 con 0x7fd8440c6e00 2024-04-25T11:39:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.792+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd844068bb0 con 0x7fd8440c6e00 2024-04-25T11:39:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.793+0000 7fd840ff9640 1 -- 172.21.15.40:0/1224533245 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd834030900 con 0x7fd8440c6e00 2024-04-25T11:39:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.793+0000 7fd840ff9640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd82406c8f0 0x7fd82406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.793+0000 7fd840ff9640 1 -- 172.21.15.40:0/1224533245 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd8340ac880 con 0x7fd8440c6e00 2024-04-25T11:39:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.794+0000 7fd843fff640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd82406c8f0 0x7fd82406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.794+0000 7fd843fff640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd82406c8f0 0x7fd82406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd82c000f60 tx=0x7fd82c007b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.798+0000 7fd840ff9640 1 -- 172.21.15.40:0/1224533245 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd83407ac20 con 0x7fd8440c6e00 2024-04-25T11:39:02.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.980+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd844072910 con 0x7fd8440c6e00 2024-04-25T11:39:02.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.980+0000 7fd840ff9640 1 -- 172.21.15.40:0/1224533245 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd83407a520 con 0x7fd8440c6e00 2024-04-25T11:39:02.967 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-25T11:39:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd82406c8f0 msgr2=0x7fd82406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd82406c8f0 0x7fd82406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd82c000f60 tx=0x7fd82c007b90 comp rx=0 tx=0).stop 2024-04-25T11:39:02.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c6e00 msgr2=0x7fd8440c3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c6e00 0x7fd8440c3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd8340097a0 tx=0x7fd8340046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 shutdown_connections 2024-04-25T11:39:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fd82406c8f0 0x7fd82406edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8440c71e0 0x7fd84415b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8440c6e00 0x7fd8440c3670 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 --2- 172.21.15.40:0/1224533245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd844072b50 0x7fd8440c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 >> 172.21.15.40:0/1224533245 conn(0x7fd84406ca80 msgr2=0x7fd84406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 shutdown_connections 2024-04-25T11:39:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:02.982+0000 7fd84a2a0640 1 -- 172.21.15.40:0/1224533245 wait complete. 2024-04-25T11:39:02.980 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:39:03.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.098+0000 7f853b1ad640 1 Processor -- start 2024-04-25T11:39:03.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.098+0000 7f853b1ad640 1 -- start start 2024-04-25T11:39:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f853b1ad640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f853415b8e0 0x7f85340d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f853b1ad640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85340cbe40 con 0x7f85340cc820 2024-04-25T11:39:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f853b1ad640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85340cb880 con 0x7f8534155070 2024-04-25T11:39:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f853b1ad640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f853415be20 con 0x7f853415b8e0 2024-04-25T11:39:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f853974a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f85340cc820 0x7f85341657c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52674/0 (socket says 172.21.15.40:52674) 2024-04-25T11:39:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f853974a640 1 -- 172.21.15.40:0/3907715387 learned_addr learned my addr 172.21.15.40:0/3907715387 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f8533fff640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f853415b8e0 0x7f85340d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 195705366 0 0) 0x7f85340cbe40 con 0x7f85340cc820 2024-04-25T11:39:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f851c003520 con 0x7f85340cc820 2024-04-25T11:39:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4014380059 0 0) 0x7f851c003520 con 0x7f85340cc820 2024-04-25T11:39:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f853415b8e0 msgr2=0x7f85340d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f853415b8e0 0x7f85340d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 >> v1:172.21.15.62:6789/0 conn(0x7f8534155070 legacy=0x7f853415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85340cbb60 con 0x7f85340cc820 2024-04-25T11:39:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7f852c0019f0 con 0x7f85340cc820 2024-04-25T11:39:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 >> v1:172.21.15.40:6789/0 conn(0x7f85340cc820 legacy=0x7f85341657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f853415b8e0 0x7f851c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f851c003f60 0x7f851c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c004d40 0x7f851c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f851c00bb50 con 0x7f851c003f60 2024-04-25T11:39:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f851c00bcd0 con 0x7f851c004d40 2024-04-25T11:39:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.099+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f851c00bed0 con 0x7f853415b8e0 2024-04-25T11:39:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.100+0000 7f8533fff640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f851c003f60 0x7f851c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.100+0000 7f8538f49640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f853415b8e0 0x7f851c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.100+0000 7f853974a640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c004d40 0x7f851c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.100+0000 7f8533fff640 1 -- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f853415b8e0 msgr2=0x7f851c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.100+0000 7f8533fff640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f853415b8e0 0x7f851c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.100+0000 7f8533fff640 1 -- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c004d40 msgr2=0x7f851c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.100+0000 7f8533fff640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c004d40 0x7f851c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.100+0000 7f8533fff640 1 -- 172.21.15.40:0/3907715387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c003520 con 0x7f851c003f60 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f8533fff640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f851c003f60 0x7f851c004800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8528009a90 tx=0x7f851c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453277f053f7c8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528036070 con 0x7f851c003f60 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8528031400 con 0x7f851c003f60 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f85337fe640 1 -- 172.21.15.40:0/3907715387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528009690 con 0x7f851c003f60 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f853b1ad640 1 -- 172.21.15.40:0/3907715387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f851c003f60 msgr2=0x7f851c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f853b1ad640 1 --2- 172.21.15.40:0/3907715387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f851c003f60 0x7f851c004800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8528009a90 tx=0x7f851c005830 comp rx=0 tx=0).stop 2024-04-25T11:39:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f853974a640 1 -- 172.21.15.40:0/3907715387 reap_dead start 2024-04-25T11:39:03.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f853b1ad640 1 -- 172.21.15.40:0/3907715387 shutdown_connections 2024-04-25T11:39:03.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f853b1ad640 1 -- 172.21.15.40:0/3907715387 >> 172.21.15.40:0/3907715387 conn(0x7f853406c680 msgr2=0x7f853406e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:03.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f853b1ad640 1 -- 172.21.15.40:0/3907715387 shutdown_connections 2024-04-25T11:39:03.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.101+0000 7f853b1ad640 1 -- 172.21.15.40:0/3907715387 wait complete. 2024-04-25T11:39:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.102+0000 7f853b1ad640 1 Processor -- start 2024-04-25T11:39:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.103+0000 7f853b1ad640 1 -- start start 2024-04-25T11:39:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.103+0000 7f853b1ad640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c003f60 0x7f85340c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.103+0000 7f853b1ad640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8534159110 0x7f85340ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.103+0000 7f853b1ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853415c090 0x7f853415c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.103+0000 7f853b1ad640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85340cb4e0 con 0x7f853415c090 2024-04-25T11:39:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.103+0000 7f853b1ad640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f853415e840 con 0x7f851c003f60 2024-04-25T11:39:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.103+0000 7f853b1ad640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8534155450 con 0x7f8534159110 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.103+0000 7f853974a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853415c090 0x7f853415c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853974a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853415c090 0x7f853415c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45472/0 (socket says 172.21.15.40:45472) 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853974a640 1 -- 172.21.15.40:0/2090274892 learned_addr learned my addr 172.21.15.40:0/2090274892 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f8533fff640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8534159110 0x7f85340ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f8538f49640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c003f60 0x7f85340c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853974a640 1 -- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8534159110 msgr2=0x7f85340ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853974a640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8534159110 0x7f85340ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853974a640 1 -- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c003f60 msgr2=0x7f85340c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853974a640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c003f60 0x7f85340c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853974a640 1 -- 172.21.15.40:0/2090274892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85340cd620 con 0x7f853415c090 2024-04-25T11:39:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f8538f49640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c003f60 0x7f85340c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853974a640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853415c090 0x7f853415c540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f853406cfc0 tx=0x7f852c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f85317fa640 1 -- 172.21.15.40:0/2090274892 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c00aa40 con 0x7f853415c090 2024-04-25T11:39:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f85317fa640 1 -- 172.21.15.40:0/2090274892 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c019070 con 0x7f853415c090 2024-04-25T11:39:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85341f09c0 con 0x7f853415c090 2024-04-25T11:39:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f85317fa640 1 -- 172.21.15.40:0/2090274892 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0133f0 con 0x7f853415c090 2024-04-25T11:39:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.104+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85341f0f80 con 0x7f853415c090 2024-04-25T11:39:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.105+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85340c9a70 con 0x7f853415c090 2024-04-25T11:39:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.106+0000 7f85317fa640 1 -- 172.21.15.40:0/2090274892 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f852c013590 con 0x7f853415c090 2024-04-25T11:39:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.106+0000 7f85317fa640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f852006c920 0x7f852006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.106+0000 7f85317fa640 1 -- 172.21.15.40:0/2090274892 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f852c08be90 con 0x7f853415c090 2024-04-25T11:39:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.106+0000 7f8538f49640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f852006c920 0x7f852006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.107+0000 7f8538f49640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f852006c920 0x7f852006ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f85240036b0 tx=0x7f8524009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.110+0000 7f85317fa640 1 -- 172.21.15.40:0/2090274892 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85340c9a70 con 0x7f853415c090 2024-04-25T11:39:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.297+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f85340ca900 con 0x7f853415c090 2024-04-25T11:39:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.297+0000 7f85317fa640 1 -- 172.21.15.40:0/2090274892 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f85340ca900 con 0x7f853415c090 2024-04-25T11:39:03.284 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-25T11:39:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f852006c920 msgr2=0x7f852006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f852006c920 0x7f852006ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f85240036b0 tx=0x7f8524009040 comp rx=0 tx=0).stop 2024-04-25T11:39:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853415c090 msgr2=0x7f853415c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853415c090 0x7f853415c540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f853406cfc0 tx=0x7f852c007a90 comp rx=0 tx=0).stop 2024-04-25T11:39:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 shutdown_connections 2024-04-25T11:39:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f852006c920 0x7f852006ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f853415c090 0x7f853415c540 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8534159110 0x7f85340ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 --2- 172.21.15.40:0/2090274892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c003f60 0x7f85340c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 >> 172.21.15.40:0/2090274892 conn(0x7f853406c680 msgr2=0x7f8534152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 shutdown_connections 2024-04-25T11:39:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.299+0000 7f853b1ad640 1 -- 172.21.15.40:0/2090274892 wait complete. 2024-04-25T11:39:03.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:39:03.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.413+0000 7f59b73f7640 1 Processor -- start 2024-04-25T11:39:03.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.413+0000 7f59b73f7640 1 -- start start 2024-04-25T11:39:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59b73f7640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b015b8e0 0x7f59b00d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59b73f7640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f59b00cc820 0x7f59b00d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59b73f7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b00cbe40 con 0x7f59b0155070 2024-04-25T11:39:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59b73f7640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f59b015be20 con 0x7f59b00cc820 2024-04-25T11:39:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59b73f7640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f59b00cb880 con 0x7f59b015b8e0 2024-04-25T11:39:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59b5193640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f59b0155070 0x7f59b015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52678/0 (socket says 172.21.15.40:52678) 2024-04-25T11:39:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59b5193640 1 -- 172.21.15.40:0/2191813040 learned_addr learned my addr 172.21.15.40:0/2191813040 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59b4992640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b015b8e0 0x7f59b00d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59b5994640 1 --2- 172.21.15.40:0/2191813040 >> v2:172.21.15.62:3300/0 conn(0x7f59b00cc820 0x7f59b00d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.414+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4273989992 0 0) 0x7f59b00cbe40 con 0x7f59b0155070 2024-04-25T11:39:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5998003520 con 0x7f59b0155070 2024-04-25T11:39:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3269935140 0 0) 0x7f5998003520 con 0x7f59b0155070 2024-04-25T11:39:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b015b8e0 msgr2=0x7f59b00d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b015b8e0 0x7f59b00d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 >> v2:172.21.15.62:3300/0 conn(0x7f59b00cc820 msgr2=0x7f59b00d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 --2- 172.21.15.40:0/2191813040 >> v2:172.21.15.62:3300/0 conn(0x7f59b00cc820 0x7f59b00d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b00cbb60 con 0x7f59b0155070 2024-04-25T11:39:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7f59a00019f0 con 0x7f59b0155070 2024-04-25T11:39:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 >> v1:172.21.15.40:6789/0 conn(0x7f59b0155070 legacy=0x7f59b015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59980039b0 0x7f5998003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5998004800 0x7f5998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59980051c0 0x7f599800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59b00cb880 con 0x7f5998004800 2024-04-25T11:39:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f599800bf00 con 0x7f59980039b0 2024-04-25T11:39:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f599800c0b0 con 0x7f59980051c0 2024-04-25T11:39:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b5994640 1 --2- 172.21.15.40:0/2191813040 >> v2:172.21.15.62:3300/0 conn(0x7f59b00cc820 0x7f59b00d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:39:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b4992640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b015b8e0 0x7f59b00d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:39:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b4992640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5998004800 0x7f5998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b5193640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59980039b0 0x7f5998003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b5994640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59980051c0 0x7f599800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b4992640 1 -- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59980051c0 msgr2=0x7f599800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b4992640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59980051c0 0x7f599800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b4992640 1 -- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59980039b0 msgr2=0x7f5998003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b4992640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59980039b0 0x7f5998003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.415+0000 7f59b4992640 1 -- 172.21.15.40:0/2191813040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5998003520 con 0x7f5998004800 2024-04-25T11:39:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.416+0000 7f59b4992640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5998004800 0x7f5998004c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f59b015bd50 tx=0x7f5998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595b49ce0b2921cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.416+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4036070 con 0x7f5998004800 2024-04-25T11:39:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.416+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a4004030 con 0x7f5998004800 2024-04-25T11:39:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.416+0000 7f59affff640 1 -- 172.21.15.40:0/2191813040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40097f0 con 0x7f5998004800 2024-04-25T11:39:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.417+0000 7f59b73f7640 1 -- 172.21.15.40:0/2191813040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5998004800 msgr2=0x7f5998004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.417+0000 7f59b73f7640 1 --2- 172.21.15.40:0/2191813040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5998004800 0x7f5998004c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f59b015bd50 tx=0x7f5998005830 comp rx=0 tx=0).stop 2024-04-25T11:39:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.417+0000 7f59b5994640 1 -- 172.21.15.40:0/2191813040 reap_dead start 2024-04-25T11:39:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.417+0000 7f59b73f7640 1 -- 172.21.15.40:0/2191813040 shutdown_connections 2024-04-25T11:39:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.417+0000 7f59b73f7640 1 -- 172.21.15.40:0/2191813040 >> 172.21.15.40:0/2191813040 conn(0x7f59b006c680 msgr2=0x7f59b006e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.417+0000 7f59b73f7640 1 -- 172.21.15.40:0/2191813040 shutdown_connections 2024-04-25T11:39:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.417+0000 7f59b73f7640 1 -- 172.21.15.40:0/2191813040 wait complete. 2024-04-25T11:39:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.418+0000 7f59b73f7640 1 Processor -- start 2024-04-25T11:39:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.418+0000 7f59b73f7640 1 -- start start 2024-04-25T11:39:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b73f7640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b0155a40 0x7f59b00c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b73f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0155e20 0x7f59b0156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b73f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59b01562f0 0x7f59b0157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b73f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59b00cb4e0 con 0x7f59b0155e20 2024-04-25T11:39:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b73f7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f59b015e840 con 0x7f59b01562f0 2024-04-25T11:39:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b73f7640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f59b0155450 con 0x7f59b0155a40 2024-04-25T11:39:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b4992640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0155e20 0x7f59b0156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b4992640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0155e20 0x7f59b0156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45482/0 (socket says 172.21.15.40:45482) 2024-04-25T11:39:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b4992640 1 -- 172.21.15.40:0/1651983176 learned_addr learned my addr 172.21.15.40:0/1651983176 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b5193640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b0155a40 0x7f59b00c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b5994640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59b01562f0 0x7f59b0157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b4992640 1 -- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b0155a40 msgr2=0x7f59b00c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b4992640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b0155a40 0x7f59b00c7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b4992640 1 -- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59b01562f0 msgr2=0x7f59b0157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b4992640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59b01562f0 0x7f59b0157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.419+0000 7f59b4992640 1 -- 172.21.15.40:0/1651983176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b01f10a0 con 0x7f59b0155e20 2024-04-25T11:39:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.420+0000 7f59b4992640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0155e20 0x7f59b0156c40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009c10 tx=0x7f59a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.420+0000 7f59adffb640 1 -- 172.21.15.40:0/1651983176 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4041030 con 0x7f59b0155e20 2024-04-25T11:39:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.420+0000 7f59adffb640 1 -- 172.21.15.40:0/1651983176 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a4003790 con 0x7f59b0155e20 2024-04-25T11:39:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.420+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b01f1380 con 0x7f59b0155e20 2024-04-25T11:39:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.420+0000 7f59adffb640 1 -- 172.21.15.40:0/1651983176 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4036040 con 0x7f59b0155e20 2024-04-25T11:39:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.420+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b01f1940 con 0x7f59b0155e20 2024-04-25T11:39:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.421+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b00c9a70 con 0x7f59b0155e20 2024-04-25T11:39:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.421+0000 7f59adffb640 1 -- 172.21.15.40:0/1651983176 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59a4044070 con 0x7f59b0155e20 2024-04-25T11:39:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.422+0000 7f59adffb640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f599c06c920 0x7f599c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.422+0000 7f59adffb640 1 -- 172.21.15.40:0/1651983176 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f59a40abb60 con 0x7f59b0155e20 2024-04-25T11:39:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.422+0000 7f59b5193640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f599c06c920 0x7f599c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.422+0000 7f59b5193640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f599c06c920 0x7f599c06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f59a0004bc0 tx=0x7f59a0006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.426+0000 7f59adffb640 1 -- 172.21.15.40:0/1651983176 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59b00c9a70 con 0x7f59b0155e20 2024-04-25T11:39:03.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.612+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f59b0068bb0 con 0x7f59b0155e20 2024-04-25T11:39:03.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.612+0000 7f59adffb640 1 -- 172.21.15.40:0/1651983176 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f59a40798b0 con 0x7f59b0155e20 2024-04-25T11:39:03.629 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-25T11:39:03.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f599c06c920 msgr2=0x7f599c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f599c06c920 0x7f599c06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f59a0004bc0 tx=0x7f59a0006040 comp rx=0 tx=0).stop 2024-04-25T11:39:03.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0155e20 msgr2=0x7f59b0156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0155e20 0x7f59b0156c40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009c10 tx=0x7f59a4004730 comp rx=0 tx=0).stop 2024-04-25T11:39:03.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 shutdown_connections 2024-04-25T11:39:03.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f599c06c920 0x7f599c06ede0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59b01562f0 0x7f59b0157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0155e20 0x7f59b0156c40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 --2- 172.21.15.40:0/1651983176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b0155a40 0x7f59b00c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 >> 172.21.15.40:0/1651983176 conn(0x7f59b006c680 msgr2=0x7f59b00ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:03.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 shutdown_connections 2024-04-25T11:39:03.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.615+0000 7f59b73f7640 1 -- 172.21.15.40:0/1651983176 wait complete. 2024-04-25T11:39:03.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:39:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.749+0000 7ff64d99b640 1 Processor -- start 2024-04-25T11:39:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.749+0000 7ff64d99b640 1 -- start start 2024-04-25T11:39:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff64d99b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff648155080 0x7ff6480cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff64d99b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff64815ed40 0x7ff64815b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff64d99b640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64815bc90 con 0x7ff6480c7840 2024-04-25T11:39:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff64d99b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff6480cb460 con 0x7ff648155080 2024-04-25T11:39:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff64d99b640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ff6480cb740 con 0x7ff64815ed40 2024-04-25T11:39:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff646ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff648155080 0x7ff6480cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff646ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff648155080 0x7ff6480cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45494/0 (socket says 172.21.15.40:45494) 2024-04-25T11:39:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff646ffd640 1 -- 172.21.15.40:0/817217184 learned_addr learned my addr 172.21.15.40:0/817217184 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:03.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff6467fc640 1 --2- 172.21.15.40:0/817217184 >> v2:172.21.15.62:3300/0 conn(0x7ff64815ed40 0x7ff64815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff646ffd640 1 -- 172.21.15.40:0/817217184 >> v2:172.21.15.62:3300/0 conn(0x7ff64815ed40 msgr2=0x7ff64815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff646ffd640 1 --2- 172.21.15.40:0/817217184 >> v2:172.21.15.62:3300/0 conn(0x7ff64815ed40 0x7ff64815b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff646ffd640 1 -- 172.21.15.40:0/817217184 >> v1:172.21.15.62:6789/0 conn(0x7ff6480c7840 legacy=0x7ff6480d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.750+0000 7ff646ffd640 1 -- 172.21.15.40:0/817217184 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64815af50 con 0x7ff648155080 2024-04-25T11:39:03.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.751+0000 7ff646ffd640 1 --2- 172.21.15.40:0/817217184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 0x7ff6480cc720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6480d1360 tx=0x7ff64002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4521c8b8486b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.751+0000 7ff645ffb640 1 -- 172.21.15.40:0/817217184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640037070 con 0x7ff648155080 2024-04-25T11:39:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.751+0000 7ff645ffb640 1 -- 172.21.15.40:0/817217184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff640032440 con 0x7ff648155080 2024-04-25T11:39:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.751+0000 7ff645ffb640 1 -- 172.21.15.40:0/817217184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640032810 con 0x7ff648155080 2024-04-25T11:39:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.752+0000 7ff64d99b640 1 -- 172.21.15.40:0/817217184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 msgr2=0x7ff6480cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.752+0000 7ff64d99b640 1 --2- 172.21.15.40:0/817217184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 0x7ff6480cc720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6480d1360 tx=0x7ff64002a160 comp rx=0 tx=0).stop 2024-04-25T11:39:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.752+0000 7ff64d99b640 1 -- 172.21.15.40:0/817217184 shutdown_connections 2024-04-25T11:39:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.752+0000 7ff64d99b640 1 --2- 172.21.15.40:0/817217184 >> v2:172.21.15.62:3300/0 conn(0x7ff64815ed40 0x7ff64815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.752+0000 7ff64d99b640 1 --2- 172.21.15.40:0/817217184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 0x7ff6480cc720 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.752+0000 7ff64d99b640 1 -- 172.21.15.40:0/817217184 >> 172.21.15.40:0/817217184 conn(0x7ff64806c680 msgr2=0x7ff64806e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.752+0000 7ff64d99b640 1 -- 172.21.15.40:0/817217184 shutdown_connections 2024-04-25T11:39:03.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.752+0000 7ff64d99b640 1 -- 172.21.15.40:0/817217184 wait complete. 2024-04-25T11:39:03.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.753+0000 7ff64d99b640 1 Processor -- start 2024-04-25T11:39:03.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.753+0000 7ff64d99b640 1 -- start start 2024-04-25T11:39:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff64d99b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6480c7840 0x7ff648152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff64d99b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 0x7ff6481f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff64d99b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64815ed40 0x7ff64815bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff64d99b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff64815f210 con 0x7ff648155080 2024-04-25T11:39:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff64d99b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff648155460 con 0x7ff64815ed40 2024-04-25T11:39:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff64d99b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff6480d0ae0 con 0x7ff6480c7840 2024-04-25T11:39:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6467fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 0x7ff6481f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6467fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 0x7ff6481f44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45504/0 (socket says 172.21.15.40:45504) 2024-04-25T11:39:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6467fc640 1 -- 172.21.15.40:0/3049207542 learned_addr learned my addr 172.21.15.40:0/3049207542 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff646ffd640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6480c7840 0x7ff648152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6467fc640 1 -- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6480c7840 msgr2=0x7ff648152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6477fe640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64815ed40 0x7ff64815bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6467fc640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6480c7840 0x7ff648152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6467fc640 1 -- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64815ed40 msgr2=0x7ff64815bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6467fc640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64815ed40 0x7ff64815bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6467fc640 1 -- 172.21.15.40:0/3049207542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff648152080 con 0x7ff648155080 2024-04-25T11:39:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff6467fc640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 0x7ff6481f44c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff63000c7f0 tx=0x7ff63000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.754+0000 7ff63ffff640 1 -- 172.21.15.40:0/3049207542 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630015070 con 0x7ff648155080 2024-04-25T11:39:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.755+0000 7ff63ffff640 1 -- 172.21.15.40:0/3049207542 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff630007820 con 0x7ff648155080 2024-04-25T11:39:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.755+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64815dbf0 con 0x7ff648155080 2024-04-25T11:39:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.755+0000 7ff63ffff640 1 -- 172.21.15.40:0/3049207542 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630007bd0 con 0x7ff648155080 2024-04-25T11:39:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.755+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6481f39e0 con 0x7ff648155080 2024-04-25T11:39:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.756+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff648068bb0 con 0x7ff648155080 2024-04-25T11:39:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.756+0000 7ff63ffff640 1 -- 172.21.15.40:0/3049207542 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff630018070 con 0x7ff648155080 2024-04-25T11:39:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.756+0000 7ff63ffff640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff62806c8f0 0x7ff62806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.756+0000 7ff63ffff640 1 -- 172.21.15.40:0/3049207542 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff63008b800 con 0x7ff648155080 2024-04-25T11:39:03.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.757+0000 7ff646ffd640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff62806c8f0 0x7ff62806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.757+0000 7ff646ffd640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff62806c8f0 0x7ff62806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff640034000 tx=0x7ff64002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.761+0000 7ff63ffff640 1 -- 172.21.15.40:0/3049207542 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff630059c30 con 0x7ff648155080 2024-04-25T11:39:03.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.959+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6480cbf20 con 0x7ff648155080 2024-04-25T11:39:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.959+0000 7ff63ffff640 1 -- 172.21.15.40:0/3049207542 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7ff630059530 con 0x7ff648155080 2024-04-25T11:39:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-25T11:39:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff62806c8f0 msgr2=0x7ff62806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff62806c8f0 0x7ff62806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff640034000 tx=0x7ff64002e000 comp rx=0 tx=0).stop 2024-04-25T11:39:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 msgr2=0x7ff6481f44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 0x7ff6481f44c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff63000c7f0 tx=0x7ff63000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 shutdown_connections 2024-04-25T11:39:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ff62806c8f0 0x7ff62806edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64815ed40 0x7ff64815bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff648155080 0x7ff6481f44c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 --2- 172.21.15.40:0/3049207542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6480c7840 0x7ff648152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 >> 172.21.15.40:0/3049207542 conn(0x7ff64806c680 msgr2=0x7ff6481514d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:03.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 shutdown_connections 2024-04-25T11:39:03.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:03.961+0000 7ff64d99b640 1 -- 172.21.15.40:0/3049207542 wait complete. 2024-04-25T11:39:03.959 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T11:39:03.959 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:03.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:39:04.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.129+0000 7f2e6b7cc640 1 Processor -- start 2024-04-25T11:39:04.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.129+0000 7f2e6b7cc640 1 -- start start 2024-04-25T11:39:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e6b7cc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e64154e10 0x7f2e640c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e6b7cc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e64071ac0 0x7f2e640cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e6b7cc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e640ca030 con 0x7f2e641551f0 2024-04-25T11:39:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e6b7cc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2e640c8790 con 0x7f2e64154e10 2024-04-25T11:39:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e6b7cc640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f2e640c8a50 con 0x7f2e64071ac0 2024-04-25T11:39:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e69568640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2e64154e10 0x7f2e640c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e68d67640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2e641551f0 0x7f2e640c79d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52686/0 (socket says 172.21.15.40:52686) 2024-04-25T11:39:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e68d67640 1 -- 172.21.15.40:0/3625564071 learned_addr learned my addr 172.21.15.40:0/3625564071 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e69d69640 1 --2- 172.21.15.40:0/3625564071 >> v2:172.21.15.62:3300/0 conn(0x7f2e64071ac0 0x7f2e640cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e69568640 1 -- 172.21.15.40:0/3625564071 >> v2:172.21.15.62:3300/0 conn(0x7f2e64071ac0 msgr2=0x7f2e640cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e69568640 1 --2- 172.21.15.40:0/3625564071 >> v2:172.21.15.62:3300/0 conn(0x7f2e64071ac0 0x7f2e640cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e69568640 1 -- 172.21.15.40:0/3625564071 >> v1:172.21.15.40:6789/0 conn(0x7f2e641551f0 legacy=0x7f2e640c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e69568640 1 -- 172.21.15.40:0/3625564071 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e640c8d30 con 0x7f2e64154e10 2024-04-25T11:39:04.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.130+0000 7f2e63fff640 1 -- 172.21.15.40:0/3625564071 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2269122213 0 0) 0x7f2e640ca030 con 0x7f2e641551f0 2024-04-25T11:39:04.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.131+0000 7f2e69568640 1 --2- 172.21.15.40:0/3625564071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64154e10 0x7f2e640c9af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2e64074150 tx=0x7f2e5c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f45f27ef57c26b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.131+0000 7f2e63fff640 1 -- 172.21.15.40:0/3625564071 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c037070 con 0x7f2e64154e10 2024-04-25T11:39:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.131+0000 7f2e63fff640 1 -- 172.21.15.40:0/3625564071 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c032440 con 0x7f2e64154e10 2024-04-25T11:39:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.131+0000 7f2e63fff640 1 -- 172.21.15.40:0/3625564071 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c032810 con 0x7f2e64154e10 2024-04-25T11:39:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.132+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/3625564071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64154e10 msgr2=0x7f2e640c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.132+0000 7f2e6b7cc640 1 --2- 172.21.15.40:0/3625564071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64154e10 0x7f2e640c9af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2e64074150 tx=0x7f2e5c02a230 comp rx=0 tx=0).stop 2024-04-25T11:39:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.132+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/3625564071 shutdown_connections 2024-04-25T11:39:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.132+0000 7f2e6b7cc640 1 --2- 172.21.15.40:0/3625564071 >> v2:172.21.15.62:3300/0 conn(0x7f2e64071ac0 0x7f2e640cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.132+0000 7f2e6b7cc640 1 --2- 172.21.15.40:0/3625564071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64154e10 0x7f2e640c9af0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.132+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/3625564071 >> 172.21.15.40:0/3625564071 conn(0x7f2e6406b970 msgr2=0x7f2e6406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.132+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/3625564071 shutdown_connections 2024-04-25T11:39:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.132+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/3625564071 wait complete. 2024-04-25T11:39:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.133+0000 7f2e6b7cc640 1 Processor -- start 2024-04-25T11:39:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.133+0000 7f2e6b7cc640 1 -- start start 2024-04-25T11:39:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e6b7cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64071ac0 0x7f2e640cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e6b7cc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e64154e10 0x7f2e64151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e6b7cc640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e6414c290 0x7f2e6414c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e6b7cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e640c95d0 con 0x7f2e64071ac0 2024-04-25T11:39:04.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e6b7cc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2e640c92f0 con 0x7f2e64154e10 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e6b7cc640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6415bad0 con 0x7f2e6414c290 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69568640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64071ac0 0x7f2e640cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69568640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64071ac0 0x7f2e640cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45526/0 (socket says 172.21.15.40:45526) 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69568640 1 -- 172.21.15.40:0/2113703810 learned_addr learned my addr 172.21.15.40:0/2113703810 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69d69640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e6414c290 0x7f2e6414c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e68d67640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e64154e10 0x7f2e64151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69568640 1 -- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e6414c290 msgr2=0x7f2e6414c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69568640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e6414c290 0x7f2e6414c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69568640 1 -- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e64154e10 msgr2=0x7f2e64151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69568640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e64154e10 0x7f2e64151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69568640 1 -- 172.21.15.40:0/2113703810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e640ccea0 con 0x7f2e64071ac0 2024-04-25T11:39:04.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e68d67640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e64154e10 0x7f2e64151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:04.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.134+0000 7f2e69568640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64071ac0 0x7f2e640cc960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c000c00 tx=0x7f2e5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.135+0000 7f2e61ffb640 1 -- 172.21.15.40:0/2113703810 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c037040 con 0x7f2e64071ac0 2024-04-25T11:39:04.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.135+0000 7f2e61ffb640 1 -- 172.21.15.40:0/2113703810 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c042070 con 0x7f2e64071ac0 2024-04-25T11:39:04.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.135+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e641f4910 con 0x7f2e64071ac0 2024-04-25T11:39:04.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.135+0000 7f2e61ffb640 1 -- 172.21.15.40:0/2113703810 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c030960 con 0x7f2e64071ac0 2024-04-25T11:39:04.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.135+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e641f4ed0 con 0x7f2e64071ac0 2024-04-25T11:39:04.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.136+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e640c6e20 con 0x7f2e64071ac0 2024-04-25T11:39:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.136+0000 7f2e61ffb640 1 -- 172.21.15.40:0/2113703810 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e5c030b00 con 0x7f2e64071ac0 2024-04-25T11:39:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.137+0000 7f2e61ffb640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2e5006c8f0 0x7f2e5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.137+0000 7f2e61ffb640 1 -- 172.21.15.40:0/2113703810 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e5c0ac3c0 con 0x7f2e64071ac0 2024-04-25T11:39:04.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.137+0000 7f2e68d67640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2e5006c8f0 0x7f2e5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.137+0000 7f2e68d67640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2e5006c8f0 0x7f2e5006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2e64151d10 tx=0x7f2e58007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.141+0000 7f2e61ffb640 1 -- 172.21.15.40:0/2113703810 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e640c6e20 con 0x7f2e64071ac0 2024-04-25T11:39:04.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.327+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e64068bb0 con 0x7f2e64071ac0 2024-04-25T11:39:04.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.328+0000 7f2e61ffb640 1 -- 172.21.15.40:0/2113703810 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f2e5c07a7f0 con 0x7f2e64071ac0 2024-04-25T11:39:04.315 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:04.315 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:38:26.664101+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:39:04.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2e5006c8f0 msgr2=0x7f2e5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2e5006c8f0 0x7f2e5006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2e64151d10 tx=0x7f2e58007c10 comp rx=0 tx=0).stop 2024-04-25T11:39:04.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64071ac0 msgr2=0x7f2e640cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64071ac0 0x7f2e640cc960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c000c00 tx=0x7f2e5c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 shutdown_connections 2024-04-25T11:39:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f2e5006c8f0 0x7f2e5006edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e6414c290 0x7f2e6414c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e64154e10 0x7f2e64151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 --2- 172.21.15.40:0/2113703810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e64071ac0 0x7f2e640cc960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 >> 172.21.15.40:0/2113703810 conn(0x7f2e6406b970 msgr2=0x7f2e64159e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 shutdown_connections 2024-04-25T11:39:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.330+0000 7f2e6b7cc640 1 -- 172.21.15.40:0/2113703810 wait complete. 2024-04-25T11:39:04.328 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:38:18.972986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:38:23.454183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:38:23.968333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:39:04.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:39:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.445+0000 7fece0a10640 1 Processor -- start 2024-04-25T11:39:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.445+0000 7fece0a10640 1 -- start start 2024-04-25T11:39:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fece0a10640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fecdc155070 0x7fecdc15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fece0a10640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15b8e0 0x7fecdc0d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fece0a10640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecdc0cbe40 con 0x7fecdc0cc820 2024-04-25T11:39:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fece0a10640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fecdc15be20 con 0x7fecdc155070 2024-04-25T11:39:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fece0a10640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fecdc0cb880 con 0x7fecdc15b8e0 2024-04-25T11:39:04.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fecdad9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fecdc0cc820 0x7fecdc0ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52698/0 (socket says 172.21.15.40:52698) 2024-04-25T11:39:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fecdad9d640 1 -- 172.21.15.40:0/4057269900 learned_addr learned my addr 172.21.15.40:0/4057269900 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fecda59c640 1 --2- 172.21.15.40:0/4057269900 >> v2:172.21.15.62:3300/0 conn(0x7fecdc155070 0x7fecdc15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fecd9d9b640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15b8e0 0x7fecdc0d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1341524299 0 0) 0x7fecdc0cbe40 con 0x7fecdc0cc820 2024-04-25T11:39:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.446+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecc8003520 con 0x7fecdc0cc820 2024-04-25T11:39:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3516277078 0 0) 0x7fecc8003520 con 0x7fecdc0cc820 2024-04-25T11:39:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15b8e0 msgr2=0x7fecdc0d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15b8e0 0x7fecdc0d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 >> v2:172.21.15.62:3300/0 conn(0x7fecdc155070 msgr2=0x7fecdc15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 --2- 172.21.15.40:0/4057269900 >> v2:172.21.15.62:3300/0 conn(0x7fecdc155070 0x7fecdc15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc0cbb60 con 0x7fecdc0cc820 2024-04-25T11:39:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7feccc0019f0 con 0x7fecdc0cc820 2024-04-25T11:39:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 >> v1:172.21.15.40:6789/0 conn(0x7fecdc0cc820 legacy=0x7fecdc0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15b8e0 0x7fecc80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc8003f30 0x7fecc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc0cb880 con 0x7fecc8004d40 2024-04-25T11:39:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fecc800b9c0 con 0x7fecc8003f30 2024-04-25T11:39:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fecc800bb90 con 0x7fecdc15b8e0 2024-04-25T11:39:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecdad9d640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecda59c640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15b8e0 0x7fecc80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecd9d9b640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc8003f30 0x7fecc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecdad9d640 1 -- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15b8e0 msgr2=0x7fecc80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecdad9d640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15b8e0 0x7fecc80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecdad9d640 1 -- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc8003f30 msgr2=0x7fecc8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecdad9d640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc8003f30 0x7fecc8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.447+0000 7fecdad9d640 1 -- 172.21.15.40:0/4057269900 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc8003520 con 0x7fecc8004d40 2024-04-25T11:39:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.448+0000 7fecdad9d640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecc80051c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feccc00a8b0 tx=0x7fecc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c221d59b6e1f8f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.448+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc00a4a0 con 0x7fecc8004d40 2024-04-25T11:39:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.448+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feccc02f070 con 0x7fecc8004d40 2024-04-25T11:39:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.448+0000 7fecd959a640 1 -- 172.21.15.40:0/4057269900 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc02bcf0 con 0x7fecc8004d40 2024-04-25T11:39:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.448+0000 7fece0a10640 1 -- 172.21.15.40:0/4057269900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 msgr2=0x7fecc80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.448+0000 7fece0a10640 1 --2- 172.21.15.40:0/4057269900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecc80051c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feccc00a8b0 tx=0x7fecc8005830 comp rx=0 tx=0).stop 2024-04-25T11:39:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.448+0000 7fecdad9d640 1 -- 172.21.15.40:0/4057269900 reap_dead start 2024-04-25T11:39:04.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.449+0000 7fece0a10640 1 -- 172.21.15.40:0/4057269900 shutdown_connections 2024-04-25T11:39:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.449+0000 7fece0a10640 1 -- 172.21.15.40:0/4057269900 >> 172.21.15.40:0/4057269900 conn(0x7fecdc06c680 msgr2=0x7fecdc06e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.449+0000 7fece0a10640 1 -- 172.21.15.40:0/4057269900 shutdown_connections 2024-04-25T11:39:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.449+0000 7fece0a10640 1 -- 172.21.15.40:0/4057269900 wait complete. 2024-04-25T11:39:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.450+0000 7fece0a10640 1 Processor -- start 2024-04-25T11:39:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.450+0000 7fece0a10640 1 -- start start 2024-04-25T11:39:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fece0a10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecdc1f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fece0a10640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0c6f80 0x7fecdc15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fece0a10640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15a940 0x7fecdc14f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fece0a10640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc0cb4e0 con 0x7fecc8004d40 2024-04-25T11:39:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fece0a10640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc15e840 con 0x7fecdc0c6f80 2024-04-25T11:39:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fece0a10640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fecdc15f2a0 con 0x7fecdc15a940 2024-04-25T11:39:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecda59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecdc1f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecda59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecdc1f08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45546/0 (socket says 172.21.15.40:45546) 2024-04-25T11:39:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecda59c640 1 -- 172.21.15.40:0/3576293060 learned_addr learned my addr 172.21.15.40:0/3576293060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecdad9d640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15a940 0x7fecdc14f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecd9d9b640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0c6f80 0x7fecdc15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecda59c640 1 -- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15a940 msgr2=0x7fecdc14f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecda59c640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15a940 0x7fecdc14f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecda59c640 1 -- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0c6f80 msgr2=0x7fecdc15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecda59c640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0c6f80 0x7fecdc15a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecda59c640 1 -- 172.21.15.40:0/3576293060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc158e20 con 0x7fecc8004d40 2024-04-25T11:39:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecdad9d640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15a940 0x7fecdc14f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecda59c640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecdc1f08b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fecd4011aa0 tx=0x7fecd4011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecd2ffd640 1 -- 172.21.15.40:0/3576293060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd400fb80 con 0x7fecc8004d40 2024-04-25T11:39:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecd2ffd640 1 -- 172.21.15.40:0/3576293060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd400fd20 con 0x7fecc8004d40 2024-04-25T11:39:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc1590e0 con 0x7fecc8004d40 2024-04-25T11:39:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.451+0000 7fecd2ffd640 1 -- 172.21.15.40:0/3576293060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4014660 con 0x7fecc8004d40 2024-04-25T11:39:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.452+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc1596a0 con 0x7fecc8004d40 2024-04-25T11:39:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.453+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecdc0c7360 con 0x7fecc8004d40 2024-04-25T11:39:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.453+0000 7fecd2ffd640 1 -- 172.21.15.40:0/3576293060 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecd4012040 con 0x7fecc8004d40 2024-04-25T11:39:04.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.453+0000 7fecd2ffd640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fecbc06c8f0 0x7fecbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.454+0000 7fecd2ffd640 1 -- 172.21.15.40:0/3576293060 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fecd408c2b0 con 0x7fecc8004d40 2024-04-25T11:39:04.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.454+0000 7fecd9d9b640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fecbc06c8f0 0x7fecbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.454+0000 7fecd9d9b640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fecbc06c8f0 0x7fecbc06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0ca8b0 tx=0x7fecc4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.458+0000 7fecd2ffd640 1 -- 172.21.15.40:0/3576293060 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fecdc0c7360 con 0x7fecc8004d40 2024-04-25T11:39:04.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.641+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fecdc068bb0 con 0x7fecc8004d40 2024-04-25T11:39:04.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.641+0000 7fecd2ffd640 1 -- 172.21.15.40:0/3576293060 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fecd4019030 con 0x7fecc8004d40 2024-04-25T11:39:04.629 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-25T11:39:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fecbc06c8f0 msgr2=0x7fecbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fecbc06c8f0 0x7fecbc06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0ca8b0 tx=0x7fecc4008040 comp rx=0 tx=0).stop 2024-04-25T11:39:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 msgr2=0x7fecdc1f08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecdc1f08b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fecd4011aa0 tx=0x7fecd4011f60 comp rx=0 tx=0).stop 2024-04-25T11:39:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 shutdown_connections 2024-04-25T11:39:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fecbc06c8f0 0x7fecbc06edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15a940 0x7fecdc14f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0c6f80 0x7fecdc15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 --2- 172.21.15.40:0/3576293060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecc8004d40 0x7fecdc1f08b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 >> 172.21.15.40:0/3576293060 conn(0x7fecdc06c680 msgr2=0x7fecdc0cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 shutdown_connections 2024-04-25T11:39:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.643+0000 7fece0a10640 1 -- 172.21.15.40:0/3576293060 wait complete. 2024-04-25T11:39:04.641 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:39:04.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.759+0000 7f950e35d640 1 Processor -- start 2024-04-25T11:39:04.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.759+0000 7f950e35d640 1 -- start start 2024-04-25T11:39:04.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.759+0000 7f950e35d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95081550f0 0x7f95080c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f950e35d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f950815e310 0x7f950815e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f950e35d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9508070010 con 0x7f95080c9990 2024-04-25T11:39:04.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f950e35d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f950815bce0 con 0x7f950815e310 2024-04-25T11:39:04.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f950e35d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f950806fd50 con 0x7f95081550f0 2024-04-25T11:39:04.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f95077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f950815e310 0x7f950815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f95077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f950815e310 0x7f950815e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45556/0 (socket says 172.21.15.40:45556) 2024-04-25T11:39:04.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f95077fe640 1 -- 172.21.15.40:0/2576700450 learned_addr learned my addr 172.21.15.40:0/2576700450 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:04.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f9507fff640 1 --2- 172.21.15.40:0/2576700450 >> v2:172.21.15.62:3300/0 conn(0x7f95081550f0 0x7f95080c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f95077fe640 1 -- 172.21.15.40:0/2576700450 >> v2:172.21.15.62:3300/0 conn(0x7f95081550f0 msgr2=0x7f95080c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:04.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f95077fe640 1 --2- 172.21.15.40:0/2576700450 >> v2:172.21.15.62:3300/0 conn(0x7f95081550f0 0x7f95080c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f95077fe640 1 -- 172.21.15.40:0/2576700450 >> v1:172.21.15.62:6789/0 conn(0x7f95080c9990 legacy=0x7f95080ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.760+0000 7f95077fe640 1 -- 172.21.15.40:0/2576700450 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95081554d0 con 0x7f950815e310 2024-04-25T11:39:04.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f95077fe640 1 --2- 172.21.15.40:0/2576700450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950815e310 0x7f950815e760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95080cbae0 tx=0x7f94fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b072ab29aebe78ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f9506ffd640 1 -- 172.21.15.40:0/2576700450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc037070 con 0x7f950815e310 2024-04-25T11:39:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f9506ffd640 1 -- 172.21.15.40:0/2576700450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94fc032440 con 0x7f950815e310 2024-04-25T11:39:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f9506ffd640 1 -- 172.21.15.40:0/2576700450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc032810 con 0x7f950815e310 2024-04-25T11:39:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f950e35d640 1 -- 172.21.15.40:0/2576700450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950815e310 msgr2=0x7f950815e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f950e35d640 1 --2- 172.21.15.40:0/2576700450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950815e310 0x7f950815e760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95080cbae0 tx=0x7f94fc02a270 comp rx=0 tx=0).stop 2024-04-25T11:39:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f950e35d640 1 -- 172.21.15.40:0/2576700450 shutdown_connections 2024-04-25T11:39:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f950e35d640 1 --2- 172.21.15.40:0/2576700450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f950815e310 0x7f950815e760 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f950e35d640 1 --2- 172.21.15.40:0/2576700450 >> v2:172.21.15.62:3300/0 conn(0x7f95081550f0 0x7f95080c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f950e35d640 1 -- 172.21.15.40:0/2576700450 >> 172.21.15.40:0/2576700450 conn(0x7f950806bfa0 msgr2=0x7f95080676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f950e35d640 1 -- 172.21.15.40:0/2576700450 shutdown_connections 2024-04-25T11:39:04.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.761+0000 7f950e35d640 1 -- 172.21.15.40:0/2576700450 wait complete. 2024-04-25T11:39:04.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.763+0000 7f950e35d640 1 Processor -- start 2024-04-25T11:39:04.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.763+0000 7f950e35d640 1 -- start start 2024-04-25T11:39:04.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950e35d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080c9990 0x7f9508149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950e35d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95081550f0 0x7f95081f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950e35d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815e310 0x7f95081f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950e35d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f950815b4e0 con 0x7f95081550f0 2024-04-25T11:39:04.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950e35d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f950815b200 con 0x7f950815e310 2024-04-25T11:39:04.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950e35d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f950815b7c0 con 0x7f95080c9990 2024-04-25T11:39:04.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f95077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95081550f0 0x7f95081f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f95077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95081550f0 0x7f95081f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45570/0 (socket says 172.21.15.40:45570) 2024-04-25T11:39:04.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f95077fe640 1 -- 172.21.15.40:0/2981259802 learned_addr learned my addr 172.21.15.40:0/2981259802 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:04.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f9507fff640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080c9990 0x7f9508149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950c8fa640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815e310 0x7f95081f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f95077fe640 1 -- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080c9990 msgr2=0x7f9508149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f95077fe640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080c9990 0x7f9508149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f95077fe640 1 -- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815e310 msgr2=0x7f95081f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f95077fe640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815e310 0x7f95081f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f95077fe640 1 -- 172.21.15.40:0/2981259802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95080cb1c0 con 0x7f95081550f0 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950c8fa640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815e310 0x7f95081f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f95077fe640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95081550f0 0x7f95081f4490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f94fc000c00 tx=0x7f94fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f9504ff9640 1 -- 172.21.15.40:0/2981259802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc041020 con 0x7f95081550f0 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f9504ff9640 1 -- 172.21.15.40:0/2981259802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94fc0306c0 con 0x7f95081550f0 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9508158d90 con 0x7f95081550f0 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f9504ff9640 1 -- 172.21.15.40:0/2981259802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc037040 con 0x7f95081550f0 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.764+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9508159350 con 0x7f95081550f0 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.765+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9508069200 con 0x7f95081550f0 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.766+0000 7f9504ff9640 1 -- 172.21.15.40:0/2981259802 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94fc044020 con 0x7f95081550f0 2024-04-25T11:39:04.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.766+0000 7f9504ff9640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f94f406c920 0x7f94f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.766+0000 7f9504ff9640 1 -- 172.21.15.40:0/2981259802 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94fc0ac320 con 0x7f95081550f0 2024-04-25T11:39:04.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.766+0000 7f9507fff640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f94f406c920 0x7f94f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.767+0000 7f9507fff640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f94f406c920 0x7f94f406ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f94f8006fd0 tx=0x7f94f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.771+0000 7f9504ff9640 1 -- 172.21.15.40:0/2981259802 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f94fc04b030 con 0x7f95081550f0 2024-04-25T11:39:04.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.955+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9508072340 con 0x7f95081550f0 2024-04-25T11:39:04.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.956+0000 7f9504ff9640 1 -- 172.21.15.40:0/2981259802 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f94fc07a750 con 0x7f95081550f0 2024-04-25T11:39:04.943 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-25T11:39:04.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.957+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f94f406c920 msgr2=0x7f94f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.957+0000 7f950e35d640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f94f406c920 0x7f94f406ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f94f8006fd0 tx=0x7f94f8008040 comp rx=0 tx=0).stop 2024-04-25T11:39:04.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.957+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95081550f0 msgr2=0x7f95081f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.957+0000 7f950e35d640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95081550f0 0x7f95081f4490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f94fc000c00 tx=0x7f94fc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.958+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 shutdown_connections 2024-04-25T11:39:04.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.958+0000 7f950e35d640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f94f406c920 0x7f94f406ede0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.958+0000 7f950e35d640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f950815e310 0x7f95081f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.958+0000 7f950e35d640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95081550f0 0x7f95081f4490 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.958+0000 7f950e35d640 1 --2- 172.21.15.40:0/2981259802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95080c9990 0x7f9508149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.958+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 >> 172.21.15.40:0/2981259802 conn(0x7f950806bfa0 msgr2=0x7f950815cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:04.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.958+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 shutdown_connections 2024-04-25T11:39:04.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:04.958+0000 7f950e35d640 1 -- 172.21.15.40:0/2981259802 wait complete. 2024-04-25T11:39:04.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:39:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.072+0000 7f96b2b20640 1 Processor -- start 2024-04-25T11:39:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.072+0000 7f96b2b20640 1 -- start start 2024-04-25T11:39:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96b2b20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96ac0cc820 0x7f96ac1667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96b2b20640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ac0cbe40 con 0x7f96ac15b8e0 2024-04-25T11:39:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96b2b20640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ac0cb880 con 0x7f96ac155070 2024-04-25T11:39:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96b2b20640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f96ac15be20 con 0x7f96ac0cc820 2024-04-25T11:39:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96b10bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96ac0cc820 0x7f96ac1667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96abfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f96ac15b8e0 0x7f96ac0d04f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52714/0 (socket says 172.21.15.40:52714) 2024-04-25T11:39:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96abfff640 1 -- 172.21.15.40:0/736748149 learned_addr learned my addr 172.21.15.40:0/736748149 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96b10bd640 1 -- 172.21.15.40:0/736748149 >> v1:172.21.15.62:6789/0 conn(0x7f96ac155070 legacy=0x7f96ac15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96b10bd640 1 -- 172.21.15.40:0/736748149 >> v1:172.21.15.40:6789/0 conn(0x7f96ac15b8e0 legacy=0x7f96ac0d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.073+0000 7f96b10bd640 1 -- 172.21.15.40:0/736748149 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac0cbb60 con 0x7f96ac0cc820 2024-04-25T11:39:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.074+0000 7f96b10bd640 1 --2- 172.21.15.40:0/736748149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0cc820 0x7f96ac1667d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009d90 tx=0x7f96a402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd612f3ef947b4a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.074+0000 7f96ab7fe640 1 -- 172.21.15.40:0/736748149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4007740 con 0x7f96ac0cc820 2024-04-25T11:39:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.074+0000 7f96ab7fe640 1 -- 172.21.15.40:0/736748149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a40078e0 con 0x7f96ac0cc820 2024-04-25T11:39:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.074+0000 7f96ab7fe640 1 -- 172.21.15.40:0/736748149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4007cb0 con 0x7f96ac0cc820 2024-04-25T11:39:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.075+0000 7f96b2b20640 1 -- 172.21.15.40:0/736748149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0cc820 msgr2=0x7f96ac1667d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.075+0000 7f96b2b20640 1 --2- 172.21.15.40:0/736748149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0cc820 0x7f96ac1667d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009d90 tx=0x7f96a402a660 comp rx=0 tx=0).stop 2024-04-25T11:39:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.075+0000 7f96b2b20640 1 -- 172.21.15.40:0/736748149 shutdown_connections 2024-04-25T11:39:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.075+0000 7f96b2b20640 1 --2- 172.21.15.40:0/736748149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac0cc820 0x7f96ac1667d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.075+0000 7f96b2b20640 1 -- 172.21.15.40:0/736748149 >> 172.21.15.40:0/736748149 conn(0x7f96ac06c680 msgr2=0x7f96ac06e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.075+0000 7f96b2b20640 1 -- 172.21.15.40:0/736748149 shutdown_connections 2024-04-25T11:39:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.075+0000 7f96b2b20640 1 -- 172.21.15.40:0/736748149 wait complete. 2024-04-25T11:39:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.076+0000 7f96b2b20640 1 Processor -- start 2024-04-25T11:39:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.076+0000 7f96b2b20640 1 -- start start 2024-04-25T11:39:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b2b20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96ac0cc820 0x7f96ac0c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b2b20640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96ac155070 0x7f96ac0c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b2b20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac15b8e0 0x7f96ac153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b2b20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac0cb4e0 con 0x7f96ac15b8e0 2024-04-25T11:39:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b2b20640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac15e840 con 0x7f96ac0cc820 2024-04-25T11:39:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b2b20640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac155450 con 0x7f96ac155070 2024-04-25T11:39:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b10bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac15b8e0 0x7f96ac153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b10bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac15b8e0 0x7f96ac153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45588/0 (socket says 172.21.15.40:45588) 2024-04-25T11:39:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b10bd640 1 -- 172.21.15.40:0/1602310630 learned_addr learned my addr 172.21.15.40:0/1602310630 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96abfff640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96ac155070 0x7f96ac0c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b08bc640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96ac0cc820 0x7f96ac0c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b10bd640 1 -- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96ac155070 msgr2=0x7f96ac0c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b10bd640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96ac155070 0x7f96ac0c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b10bd640 1 -- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96ac0cc820 msgr2=0x7f96ac0c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b10bd640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96ac0cc820 0x7f96ac0c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b10bd640 1 -- 172.21.15.40:0/1602310630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac153ab0 con 0x7f96ac15b8e0 2024-04-25T11:39:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96b10bd640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac15b8e0 0x7f96ac153070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009eb0 tx=0x7f96a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.077+0000 7f96a97fa640 1 -- 172.21.15.40:0/1602310630 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a403f070 con 0x7f96ac15b8e0 2024-04-25T11:39:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.078+0000 7f96a97fa640 1 -- 172.21.15.40:0/1602310630 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a4031c90 con 0x7f96ac15b8e0 2024-04-25T11:39:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.078+0000 7f96a97fa640 1 -- 172.21.15.40:0/1602310630 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4044680 con 0x7f96ac15b8e0 2024-04-25T11:39:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.078+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac153d90 con 0x7f96ac15b8e0 2024-04-25T11:39:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.078+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac1e5240 con 0x7f96ac15b8e0 2024-04-25T11:39:05.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.079+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ac0cabb0 con 0x7f96ac15b8e0 2024-04-25T11:39:05.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.079+0000 7f96a97fa640 1 -- 172.21.15.40:0/1602310630 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96a403b410 con 0x7f96ac15b8e0 2024-04-25T11:39:05.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.079+0000 7f96a97fa640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f969806c8f0 0x7f969806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.080+0000 7f96a97fa640 1 -- 172.21.15.40:0/1602310630 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f96a40ad410 con 0x7f96ac15b8e0 2024-04-25T11:39:05.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.080+0000 7f96b08bc640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f969806c8f0 0x7f969806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.080+0000 7f96b08bc640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f969806c8f0 0x7f969806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f969c007fd0 tx=0x7f969c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:05.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.084+0000 7f96a97fa640 1 -- 172.21.15.40:0/1602310630 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96ac0cabb0 con 0x7f96ac15b8e0 2024-04-25T11:39:05.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.271+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f96ac073b30 con 0x7f96ac15b8e0 2024-04-25T11:39:05.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.271+0000 7f96a97fa640 1 -- 172.21.15.40:0/1602310630 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f96ac073b30 con 0x7f96ac15b8e0 2024-04-25T11:39:05.259 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-25T11:39:05.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f969806c8f0 msgr2=0x7f969806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f969806c8f0 0x7f969806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f969c007fd0 tx=0x7f969c009040 comp rx=0 tx=0).stop 2024-04-25T11:39:05.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac15b8e0 msgr2=0x7f96ac153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac15b8e0 0x7f96ac153070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009eb0 tx=0x7f96a4004a40 comp rx=0 tx=0).stop 2024-04-25T11:39:05.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 shutdown_connections 2024-04-25T11:39:05.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f969806c8f0 0x7f969806edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96ac15b8e0 0x7f96ac153070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96ac155070 0x7f96ac0c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 --2- 172.21.15.40:0/1602310630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96ac0cc820 0x7f96ac0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 >> 172.21.15.40:0/1602310630 conn(0x7f96ac06c680 msgr2=0x7f96ac159250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:05.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 shutdown_connections 2024-04-25T11:39:05.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.273+0000 7f96b2b20640 1 -- 172.21.15.40:0/1602310630 wait complete. 2024-04-25T11:39:05.271 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:39:05.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.385+0000 7f65af5cd640 1 Processor -- start 2024-04-25T11:39:05.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.386+0000 7f65af5cd640 1 -- start start 2024-04-25T11:39:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.386+0000 7f65af5cd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65a80c51c0 0x7f65a80c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.386+0000 7f65af5cd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65a806fee0 0x7f65a80702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.386+0000 7f65af5cd640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a80c9640 con 0x7f65a8156130 2024-04-25T11:39:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.386+0000 7f65af5cd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f65a80ca2e0 con 0x7f65a80c51c0 2024-04-25T11:39:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.386+0000 7f65af5cd640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f65a80ca5c0 con 0x7f65a806fee0 2024-04-25T11:39:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.387+0000 7f65acb68640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65a80c51c0 0x7f65a80c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.387+0000 7f65ad369640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f65a8156130 0x7f65a815c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52724/0 (socket says 172.21.15.40:52724) 2024-04-25T11:39:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.387+0000 7f65ad369640 1 -- 172.21.15.40:0/4193087775 learned_addr learned my addr 172.21.15.40:0/4193087775 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.387+0000 7f65adb6a640 1 --2- 172.21.15.40:0/4193087775 >> v2:172.21.15.62:3300/0 conn(0x7f65a806fee0 0x7f65a80702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.387+0000 7f65acb68640 1 -- 172.21.15.40:0/4193087775 >> v2:172.21.15.62:3300/0 conn(0x7f65a806fee0 msgr2=0x7f65a80702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.387+0000 7f65acb68640 1 --2- 172.21.15.40:0/4193087775 >> v2:172.21.15.62:3300/0 conn(0x7f65a806fee0 0x7f65a80702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.387+0000 7f65acb68640 1 -- 172.21.15.40:0/4193087775 >> v1:172.21.15.40:6789/0 conn(0x7f65a8156130 legacy=0x7f65a815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.387+0000 7f659ffff640 1 -- 172.21.15.40:0/4193087775 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1809573408 0 0) 0x7f65a80c9640 con 0x7f65a8156130 2024-04-25T11:39:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.387+0000 7f65acb68640 1 -- 172.21.15.40:0/4193087775 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a80ca8a0 con 0x7f65a80c51c0 2024-04-25T11:39:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f65acb68640 1 --2- 172.21.15.40:0/4193087775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a80c51c0 0x7f65a80c8d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65a80b3520 tx=0x7f659802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88af062f92c46ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:05.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f659ffff640 1 -- 172.21.15.40:0/4193087775 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598037070 con 0x7f65a80c51c0 2024-04-25T11:39:05.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f659ffff640 1 -- 172.21.15.40:0/4193087775 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6598032440 con 0x7f65a80c51c0 2024-04-25T11:39:05.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f659ffff640 1 -- 172.21.15.40:0/4193087775 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598032810 con 0x7f65a80c51c0 2024-04-25T11:39:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f65af5cd640 1 -- 172.21.15.40:0/4193087775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a80c51c0 msgr2=0x7f65a80c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f65af5cd640 1 --2- 172.21.15.40:0/4193087775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a80c51c0 0x7f65a80c8d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65a80b3520 tx=0x7f659802a230 comp rx=0 tx=0).stop 2024-04-25T11:39:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f65af5cd640 1 -- 172.21.15.40:0/4193087775 shutdown_connections 2024-04-25T11:39:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f65af5cd640 1 --2- 172.21.15.40:0/4193087775 >> v2:172.21.15.62:3300/0 conn(0x7f65a806fee0 0x7f65a80702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f65af5cd640 1 --2- 172.21.15.40:0/4193087775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a80c51c0 0x7f65a80c8d50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f65af5cd640 1 -- 172.21.15.40:0/4193087775 >> 172.21.15.40:0/4193087775 conn(0x7f65a806bd90 msgr2=0x7f65a80676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f65af5cd640 1 -- 172.21.15.40:0/4193087775 shutdown_connections 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.388+0000 7f65af5cd640 1 -- 172.21.15.40:0/4193087775 wait complete. 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.389+0000 7f65af5cd640 1 Processor -- start 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65af5cd640 1 -- start start 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65af5cd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a806fee0 0x7f65a8149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65af5cd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65a80c51c0 0x7f65a81569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65af5cd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a8156130 0x7f65a814e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65af5cd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f65a80cbb90 con 0x7f65a806fee0 2024-04-25T11:39:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65af5cd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65a80ce640 con 0x7f65a8156130 2024-04-25T11:39:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65af5cd640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f65a80c9290 con 0x7f65a80c51c0 2024-04-25T11:39:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65ad369640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a806fee0 0x7f65a8149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65ad369640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a806fee0 0x7f65a8149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45614/0 (socket says 172.21.15.40:45614) 2024-04-25T11:39:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.390+0000 7f65ad369640 1 -- 172.21.15.40:0/4100129217 learned_addr learned my addr 172.21.15.40:0/4100129217 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65adb6a640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a8156130 0x7f65a814e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65acb68640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65a80c51c0 0x7f65a81569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65ad369640 1 -- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65a80c51c0 msgr2=0x7f65a81569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65ad369640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65a80c51c0 0x7f65a81569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65ad369640 1 -- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a8156130 msgr2=0x7f65a814e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65ad369640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a8156130 0x7f65a814e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65ad369640 1 -- 172.21.15.40:0/4100129217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a80b2f50 con 0x7f65a806fee0 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65acb68640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65a80c51c0 0x7f65a81569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65adb6a640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a8156130 0x7f65a814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65ad369640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a806fee0 0x7f65a8149060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f65a400c430 tx=0x7f65a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f659dffb640 1 -- 172.21.15.40:0/4100129217 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40074e0 con 0x7f65a806fee0 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f659dffb640 1 -- 172.21.15.40:0/4100129217 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a40047e0 con 0x7f65a806fee0 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a81eebe0 con 0x7f65a806fee0 2024-04-25T11:39:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f659dffb640 1 -- 172.21.15.40:0/4100129217 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4004bd0 con 0x7f65a806fee0 2024-04-25T11:39:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.391+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a81eee80 con 0x7f65a806fee0 2024-04-25T11:39:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.392+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a8069200 con 0x7f65a806fee0 2024-04-25T11:39:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.392+0000 7f659dffb640 1 -- 172.21.15.40:0/4100129217 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65a4011070 con 0x7f65a806fee0 2024-04-25T11:39:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.393+0000 7f659dffb640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f658c06c8f0 0x7f658c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.393+0000 7f659dffb640 1 -- 172.21.15.40:0/4100129217 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f65a408b750 con 0x7f65a806fee0 2024-04-25T11:39:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.393+0000 7f65acb68640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f658c06c8f0 0x7f658c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.393+0000 7f65acb68640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f658c06c8f0 0x7f658c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6598034000 tx=0x7f659802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:05.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.397+0000 7f659dffb640 1 -- 172.21.15.40:0/4100129217 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65a80b2f50 con 0x7f65a806fee0 2024-04-25T11:39:05.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.595+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f65a81542f0 con 0x7f65a806fee0 2024-04-25T11:39:05.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.595+0000 7f659dffb640 1 -- 172.21.15.40:0/4100129217 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f65a4059b80 con 0x7f65a806fee0 2024-04-25T11:39:05.582 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-25T11:39:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f658c06c8f0 msgr2=0x7f658c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f658c06c8f0 0x7f658c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6598034000 tx=0x7f659802e000 comp rx=0 tx=0).stop 2024-04-25T11:39:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a806fee0 msgr2=0x7f65a8149060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a806fee0 0x7f65a8149060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f65a400c430 tx=0x7f65a400c8f0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 shutdown_connections 2024-04-25T11:39:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f658c06c8f0 0x7f658c06edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65a8156130 0x7f65a814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65a80c51c0 0x7f65a81569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 --2- 172.21.15.40:0/4100129217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65a806fee0 0x7f65a8149060 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 >> 172.21.15.40:0/4100129217 conn(0x7f65a806bd90 msgr2=0x7f65a80c8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:05.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 shutdown_connections 2024-04-25T11:39:05.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:05.597+0000 7f65af5cd640 1 -- 172.21.15.40:0/4100129217 wait complete. 2024-04-25T11:39:05.595 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi040.front.sepia.ceph.com... 2024-04-25T11:39:05.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:05.595 DEBUG:teuthology.orchestra.run.smithi040:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:05.618 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[42246]: fuse finished with error 0 and tester_r 0 2024-04-25T11:39:05.620 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:39:11.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:39:11.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:11.722 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:39:11.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:11.749 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-25T11:39:11.809 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:39:11.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:11.809 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:39:11.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:11.879 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-25T11:39:11.989 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T11:39:11.998 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:39:11.998 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:39:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.116+0000 7f92032a0640 1 Processor -- start 2024-04-25T11:39:12.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.116+0000 7f92032a0640 1 -- start start 2024-04-25T11:39:12.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f92032a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91fc151bc0 0x7f91fc158500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f92032a0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91fc151fa0 0x7f91fc0c5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f92032a0640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 0x7f91fc0722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f92032a0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f91fc158a40 con 0x7f91fc151bc0 2024-04-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f92032a0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f91fc0c3930 con 0x7f91fc151fa0 2024-04-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f92032a0640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f91fc0c3c10 con 0x7f91fc0c5a50 2024-04-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920103c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91fc151bc0 0x7f91fc158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920103c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91fc151bc0 0x7f91fc158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45616/0 (socket says 172.21.15.40:45616) 2024-04-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920103c640 1 -- 172.21.15.40:0/3001606126 learned_addr learned my addr 172.21.15.40:0/3001606126 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920183d640 1 --2- 172.21.15.40:0/3001606126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 0x7f91fc0722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920103c640 1 -- 172.21.15.40:0/3001606126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 msgr2=0x7f91fc0722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920083b640 1 --2- 172.21.15.40:0/3001606126 >> v2:172.21.15.62:3300/0 conn(0x7f91fc151fa0 0x7f91fc0c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920103c640 1 --2- 172.21.15.40:0/3001606126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 0x7f91fc0722d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920103c640 1 -- 172.21.15.40:0/3001606126 >> v2:172.21.15.62:3300/0 conn(0x7f91fc151fa0 msgr2=0x7f91fc0c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:12.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920103c640 1 --2- 172.21.15.40:0/3001606126 >> v2:172.21.15.62:3300/0 conn(0x7f91fc151fa0 0x7f91fc0c5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920103c640 1 -- 172.21.15.40:0/3001606126 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc0717f0 con 0x7f91fc151bc0 2024-04-25T11:39:12.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920183d640 1 --2- 172.21.15.40:0/3001606126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 0x7f91fc0722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:12.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.117+0000 7f920083b640 1 --2- 172.21.15.40:0/3001606126 >> v2:172.21.15.62:3300/0 conn(0x7f91fc151fa0 0x7f91fc0c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:12.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.118+0000 7f920103c640 1 --2- 172.21.15.40:0/3001606126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 0x7f91fc158500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f91fc06c950 tx=0x7f91f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b92a942c502f6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.118+0000 7f91fbfff640 1 -- 172.21.15.40:0/3001606126 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4037070 con 0x7f91fc151bc0 2024-04-25T11:39:12.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.118+0000 7f91fbfff640 1 -- 172.21.15.40:0/3001606126 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f4032440 con 0x7f91fc151bc0 2024-04-25T11:39:12.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.118+0000 7f91fbfff640 1 -- 172.21.15.40:0/3001606126 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4032810 con 0x7f91fc151bc0 2024-04-25T11:39:12.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.119+0000 7f92032a0640 1 -- 172.21.15.40:0/3001606126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 msgr2=0x7f91fc158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.119+0000 7f92032a0640 1 --2- 172.21.15.40:0/3001606126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 0x7f91fc158500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f91fc06c950 tx=0x7f91f402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.119+0000 7f92032a0640 1 -- 172.21.15.40:0/3001606126 shutdown_connections 2024-04-25T11:39:12.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.119+0000 7f92032a0640 1 --2- 172.21.15.40:0/3001606126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 0x7f91fc0722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.119+0000 7f92032a0640 1 --2- 172.21.15.40:0/3001606126 >> v2:172.21.15.62:3300/0 conn(0x7f91fc151fa0 0x7f91fc0c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.119+0000 7f92032a0640 1 --2- 172.21.15.40:0/3001606126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 0x7f91fc158500 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.119+0000 7f92032a0640 1 -- 172.21.15.40:0/3001606126 >> 172.21.15.40:0/3001606126 conn(0x7f91fc06b970 msgr2=0x7f91fc06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:12.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.119+0000 7f92032a0640 1 -- 172.21.15.40:0/3001606126 shutdown_connections 2024-04-25T11:39:12.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.119+0000 7f92032a0640 1 -- 172.21.15.40:0/3001606126 wait complete. 2024-04-25T11:39:12.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.120+0000 7f92032a0640 1 Processor -- start 2024-04-25T11:39:12.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.120+0000 7f92032a0640 1 -- start start 2024-04-25T11:39:12.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f92032a0640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 0x7f91fc0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f92032a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 0x7f91fc15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f92032a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc151fa0 0x7f91fc145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f92032a0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc0c3ef0 con 0x7f91fc151bc0 2024-04-25T11:39:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f92032a0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc071db0 con 0x7f91fc151fa0 2024-04-25T11:39:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f92032a0640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f91fc06ff20 con 0x7f91fc0c5a50 2024-04-25T11:39:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920083b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 0x7f91fc15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920083b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 0x7f91fc15ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45618/0 (socket says 172.21.15.40:45618) 2024-04-25T11:39:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920083b640 1 -- 172.21.15.40:0/1919792248 learned_addr learned my addr 172.21.15.40:0/1919792248 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920183d640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc151fa0 0x7f91fc145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920103c640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 0x7f91fc0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920083b640 1 -- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 msgr2=0x7f91fc0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920083b640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 0x7f91fc0ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920083b640 1 -- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc151fa0 msgr2=0x7f91fc145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920083b640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc151fa0 0x7f91fc145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920083b640 1 -- 172.21.15.40:0/1919792248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc0cbae0 con 0x7f91fc151bc0 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f920083b640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 0x7f91fc15ae40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c820 tx=0x7f91f000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.121+0000 7f91f9ffb640 1 -- 172.21.15.40:0/1919792248 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0015070 con 0x7f91fc151bc0 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.122+0000 7f91f9ffb640 1 -- 172.21.15.40:0/1919792248 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f0007a30 con 0x7f91fc151bc0 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.122+0000 7f91f9ffb640 1 -- 172.21.15.40:0/1919792248 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0007e00 con 0x7f91fc151bc0 2024-04-25T11:39:12.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.122+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91fc158e90 con 0x7f91fc151bc0 2024-04-25T11:39:12.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.122+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91fc159450 con 0x7f91fc151bc0 2024-04-25T11:39:12.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.123+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91fc069cd0 con 0x7f91fc151bc0 2024-04-25T11:39:12.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.123+0000 7f91f9ffb640 1 -- 172.21.15.40:0/1919792248 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91f0018070 con 0x7f91fc151bc0 2024-04-25T11:39:12.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.123+0000 7f91f9ffb640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91e806c8f0 0x7f91e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.124+0000 7f91f9ffb640 1 -- 172.21.15.40:0/1919792248 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f91f008ba90 con 0x7f91fc151bc0 2024-04-25T11:39:12.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.124+0000 7f920103c640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91e806c8f0 0x7f91e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.124+0000 7f920103c640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91e806c8f0 0x7f91e806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91f4034000 tx=0x7f91f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.128+0000 7f91f9ffb640 1 -- 172.21.15.40:0/1919792248 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91fc069cd0 con 0x7f91fc151bc0 2024-04-25T11:39:12.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.309+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91fc158c40 con 0x7f91e806c8f0 2024-04-25T11:39:12.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.316+0000 7f91f9ffb640 1 -- 172.21.15.40:0/1919792248 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252064 (secure 0 0 0) 0x7f91fc158c40 con 0x7f91e806c8f0 2024-04-25T11:39:12.305 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:12.307 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:39:12.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91e806c8f0 msgr2=0x7f91e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91e806c8f0 0x7f91e806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91f4034000 tx=0x7f91f402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:12.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 msgr2=0x7f91fc15ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 0x7f91fc15ae40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c820 tx=0x7f91f000cc20 comp rx=0 tx=0).stop 2024-04-25T11:39:12.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 shutdown_connections 2024-04-25T11:39:12.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91e806c8f0 0x7f91e806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc151fa0 0x7f91fc145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91fc151bc0 0x7f91fc15ae40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 --2- 172.21.15.40:0/1919792248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91fc0c5a50 0x7f91fc0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 >> 172.21.15.40:0/1919792248 conn(0x7f91fc06b970 msgr2=0x7f91fc14d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:12.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 shutdown_connections 2024-04-25T11:39:12.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.319+0000 7f92032a0640 1 -- 172.21.15.40:0/1919792248 wait complete. 2024-04-25T11:39:12.317 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-25T11:39:10.959552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601545,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":377,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218112,"kb_used_data":1984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":2031616,"data_stored":1770562,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3143,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002458"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762879+0000","last_change":"2024-04-25T11:38:25.677572+0000","last_active":"2024-04-25T11:38:26.762879+0000","last_peered":"2024-04-25T11:38:26.762879+0000","last_clean":"2024-04-25T11:38:26.762879+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:38:26.762879+0000","last_undegraded":"2024-04-25T11:38:26.762879+0000","last_fullsized":"2024-04-25T11:38:26.762879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673367+0000","last_change":"2024-04-25T11:38:25.677437+0000","last_active":"2024-04-25T11:38:26.673367+0000","last_peered":"2024-04-25T11:38:26.673367+0000","last_clean":"2024-04-25T11:38:26.673367+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:38:26.673367+0000","last_undegraded":"2024-04-25T11:38:26.673367+0000","last_fullsized":"2024-04-25T11:38:26.673367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761847+0000","last_change":"2024-04-25T11:38:25.676553+0000","last_active":"2024-04-25T11:38:26.761847+0000","last_peered":"2024-04-25T11:38:26.761847+0000","last_clean":"2024-04-25T11:38:26.761847+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:38:26.761847+0000","last_undegraded":"2024-04-25T11:38:26.761847+0000","last_fullsized":"2024-04-25T11:38:26.761847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762862+0000","last_change":"2024-04-25T11:38:25.676518+0000","last_active":"2024-04-25T11:38:26.762862+0000","last_peered":"2024-04-25T11:38:26.762862+0000","last_clean":"2024-04-25T11:38:26.762862+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:38:26.762862+0000","last_undegraded":"2024-04-25T11:38:26.762862+0000","last_fullsized":"2024-04-25T11:38:26.762862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762062+0000","last_change":"2024-04-25T11:38:25.678524+0000","last_active":"2024-04-25T11:38:26.762062+0000","last_peered":"2024-04-25T11:38:26.762062+0000","last_clean":"2024-04-25T11:38:26.762062+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:38:26.762062+0000","last_undegraded":"2024-04-25T11:38:26.762062+0000","last_fullsized":"2024-04-25T11:38:26.762062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761774+0000","last_change":"2024-04-25T11:38:25.683983+0000","last_active":"2024-04-25T11:38:26.761774+0000","last_peered":"2024-04-25T11:38:26.761774+0000","last_clean":"2024-04-25T11:38:26.761774+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:38:26.761774+0000","last_undegraded":"2024-04-25T11:38:26.761774+0000","last_fullsized":"2024-04-25T11:38:26.761774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761720+0000","last_change":"2024-04-25T11:38:25.682904+0000","last_active":"2024-04-25T11:38:26.761720+0000","last_peered":"2024-04-25T11:38:26.761720+0000","last_clean":"2024-04-25T11:38:26.761720+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:38:26.761720+0000","last_undegraded":"2024-04-25T11:38:26.761720+0000","last_fullsized":"2024-04-25T11:38:26.761720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761274+0000","last_change":"2024-04-25T11:38:25.670312+0000","last_active":"2024-04-25T11:38:26.761274+0000","last_peered":"2024-04-25T11:38:26.761274+0000","last_clean":"2024-04-25T11:38:26.761274+0000","last_became_active":"2024-04-25T11:38:25.669988+0000","last_became_peered":"2024-04-25T11:38:25.669988+0000","last_unstale":"2024-04-25T11:38:26.761274+0000","last_undegraded":"2024-04-25T11:38:26.761274+0000","last_fullsized":"2024-04-25T11:38:26.761274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762939+0000","last_change":"2024-04-25T11:38:25.678459+0000","last_active":"2024-04-25T11:38:26.762939+0000","last_peered":"2024-04-25T11:38:26.762939+0000","last_clean":"2024-04-25T11:38:26.762939+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:38:26.762939+0000","last_undegraded":"2024-04-25T11:38:26.762939+0000","last_fullsized":"2024-04-25T11:38:26.762939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674629+0000","last_change":"2024-04-25T11:38:25.675561+0000","last_active":"2024-04-25T11:38:26.674629+0000","last_peered":"2024-04-25T11:38:26.674629+0000","last_clean":"2024-04-25T11:38:26.674629+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:38:26.674629+0000","last_undegraded":"2024-04-25T11:38:26.674629+0000","last_fullsized":"2024-04-25T11:38:26.674629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674251+0000","last_change":"2024-04-25T11:38:25.671113+0000","last_active":"2024-04-25T11:38:26.674251+0000","last_peered":"2024-04-25T11:38:26.674251+0000","last_clean":"2024-04-25T11:38:26.674251+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:38:26.674251+0000","last_undegraded":"2024-04-25T11:38:26.674251+0000","last_fullsized":"2024-04-25T11:38:26.674251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762821+0000","last_change":"2024-04-25T11:38:25.665690+0000","last_active":"2024-04-25T11:38:26.762821+0000","last_peered":"2024-04-25T11:38:26.762821+0000","last_clean":"2024-04-25T11:38:26.762821+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:38:26.762821+0000","last_undegraded":"2024-04-25T11:38:26.762821+0000","last_fullsized":"2024-04-25T11:38:26.762821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:25.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761152+0000","last_change":"2024-04-25T11:38:25.666858+0000","last_active":"2024-04-25T11:38:26.761152+0000","last_peered":"2024-04-25T11:38:26.761152+0000","last_clean":"2024-04-25T11:38:26.761152+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:38:26.761152+0000","last_undegraded":"2024-04-25T11:38:26.761152+0000","last_fullsized":"2024-04-25T11:38:26.761152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761293+0000","last_change":"2024-04-25T11:38:25.677561+0000","last_active":"2024-04-25T11:38:26.761293+0000","last_peered":"2024-04-25T11:38:26.761293+0000","last_clean":"2024-04-25T11:38:26.761293+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:38:26.761293+0000","last_undegraded":"2024-04-25T11:38:26.761293+0000","last_fullsized":"2024-04-25T11:38:26.761293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673120+0000","last_change":"2024-04-25T11:38:25.672129+0000","last_active":"2024-04-25T11:38:26.673120+0000","last_peered":"2024-04-25T11:38:26.673120+0000","last_clean":"2024-04-25T11:38:26.673120+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:38:26.673120+0000","last_undegraded":"2024-04-25T11:38:26.673120+0000","last_fullsized":"2024-04-25T11:38:26.673120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761886+0000","last_change":"2024-04-25T11:38:25.682547+0000","last_active":"2024-04-25T11:38:26.761886+0000","last_peered":"2024-04-25T11:38:26.761886+0000","last_clean":"2024-04-25T11:38:26.761886+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:38:26.761886+0000","last_undegraded":"2024-04-25T11:38:26.761886+0000","last_fullsized":"2024-04-25T11:38:26.761886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763750+0000","last_change":"2024-04-25T11:38:25.683798+0000","last_active":"2024-04-25T11:38:26.763750+0000","last_peered":"2024-04-25T11:38:26.763750+0000","last_clean":"2024-04-25T11:38:26.763750+0000","last_became_active":"2024-04-25T11:38:25.682590+0000","last_became_peered":"2024-04-25T11:38:25.682590+0000","last_unstale":"2024-04-25T11:38:26.763750+0000","last_undegraded":"2024-04-25T11:38:26.763750+0000","last_fullsized":"2024-04-25T11:38:26.763750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673869+0000","last_change":"2024-04-25T11:38:25.669650+0000","last_active":"2024-04-25T11:38:26.673869+0000","last_peered":"2024-04-25T11:38:26.673869+0000","last_clean":"2024-04-25T11:38:26.673869+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:38:26.673869+0000","last_undegraded":"2024-04-25T11:38:26.673869+0000","last_fullsized":"2024-04-25T11:38:26.673869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762614+0000","last_change":"2024-04-25T11:38:25.678238+0000","last_active":"2024-04-25T11:38:26.762614+0000","last_peered":"2024-04-25T11:38:26.762614+0000","last_clean":"2024-04-25T11:38:26.762614+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:38:26.762614+0000","last_undegraded":"2024-04-25T11:38:26.762614+0000","last_fullsized":"2024-04-25T11:38:26.762614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674167+0000","last_change":"2024-04-25T11:38:25.666736+0000","last_active":"2024-04-25T11:38:26.674167+0000","last_peered":"2024-04-25T11:38:26.674167+0000","last_clean":"2024-04-25T11:38:26.674167+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:38:26.674167+0000","last_undegraded":"2024-04-25T11:38:26.674167+0000","last_fullsized":"2024-04-25T11:38:26.674167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761661+0000","last_change":"2024-04-25T11:38:25.680587+0000","last_active":"2024-04-25T11:38:26.761661+0000","last_peered":"2024-04-25T11:38:26.761661+0000","last_clean":"2024-04-25T11:38:26.761661+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:38:26.761661+0000","last_undegraded":"2024-04-25T11:38:26.761661+0000","last_fullsized":"2024-04-25T11:38:26.761661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762157+0000","last_change":"2024-04-25T11:38:25.680464+0000","last_active":"2024-04-25T11:38:26.762157+0000","last_peered":"2024-04-25T11:38:26.762157+0000","last_clean":"2024-04-25T11:38:26.762157+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:38:26.762157+0000","last_undegraded":"2024-04-25T11:38:26.762157+0000","last_fullsized":"2024-04-25T11:38:26.762157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761201+0000","last_change":"2024-04-25T11:38:25.672189+0000","last_active":"2024-04-25T11:38:26.761201+0000","last_peered":"2024-04-25T11:38:26.761201+0000","last_clean":"2024-04-25T11:38:26.761201+0000","last_became_active":"2024-04-25T11:38:25.671920+0000","last_became_peered":"2024-04-25T11:38:25.671920+0000","last_unstale":"2024-04-25T11:38:26.761201+0000","last_undegraded":"2024-04-25T11:38:26.761201+0000","last_fullsized":"2024-04-25T11:38:26.761201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762072+0000","last_change":"2024-04-25T11:38:25.674108+0000","last_active":"2024-04-25T11:38:26.762072+0000","last_peered":"2024-04-25T11:38:26.762072+0000","last_clean":"2024-04-25T11:38:26.762072+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:38:26.762072+0000","last_undegraded":"2024-04-25T11:38:26.762072+0000","last_fullsized":"2024-04-25T11:38:26.762072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761259+0000","last_change":"2024-04-25T11:38:25.672215+0000","last_active":"2024-04-25T11:38:26.761259+0000","last_peered":"2024-04-25T11:38:26.761259+0000","last_clean":"2024-04-25T11:38:26.761259+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:38:26.761259+0000","last_undegraded":"2024-04-25T11:38:26.761259+0000","last_fullsized":"2024-04-25T11:38:26.761259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674666+0000","last_change":"2024-04-25T11:38:25.679772+0000","last_active":"2024-04-25T11:38:26.674666+0000","last_peered":"2024-04-25T11:38:26.674666+0000","last_clean":"2024-04-25T11:38:26.674666+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:38:26.674666+0000","last_undegraded":"2024-04-25T11:38:26.674666+0000","last_fullsized":"2024-04-25T11:38:26.674666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673164+0000","last_change":"2024-04-25T11:38:25.665626+0000","last_active":"2024-04-25T11:38:26.673164+0000","last_peered":"2024-04-25T11:38:26.673164+0000","last_clean":"2024-04-25T11:38:26.673164+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:38:26.673164+0000","last_undegraded":"2024-04-25T11:38:26.673164+0000","last_fullsized":"2024-04-25T11:38:26.673164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761385+0000","last_change":"2024-04-25T11:38:25.672935+0000","last_active":"2024-04-25T11:38:26.761385+0000","last_peered":"2024-04-25T11:38:26.761385+0000","last_clean":"2024-04-25T11:38:26.761385+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:38:26.761385+0000","last_undegraded":"2024-04-25T11:38:26.761385+0000","last_fullsized":"2024-04-25T11:38:26.761385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761842+0000","last_change":"2024-04-25T11:38:25.679486+0000","last_active":"2024-04-25T11:38:26.761842+0000","last_peered":"2024-04-25T11:38:26.761842+0000","last_clean":"2024-04-25T11:38:26.761842+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:38:26.761842+0000","last_undegraded":"2024-04-25T11:38:26.761842+0000","last_fullsized":"2024-04-25T11:38:26.761842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762406+0000","last_change":"2024-04-25T11:38:25.679437+0000","last_active":"2024-04-25T11:38:26.762406+0000","last_peered":"2024-04-25T11:38:26.762406+0000","last_clean":"2024-04-25T11:38:26.762406+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:38:26.762406+0000","last_undegraded":"2024-04-25T11:38:26.762406+0000","last_fullsized":"2024-04-25T11:38:26.762406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673702+0000","last_change":"2024-04-25T11:38:25.670384+0000","last_active":"2024-04-25T11:38:26.673702+0000","last_peered":"2024-04-25T11:38:26.673702+0000","last_clean":"2024-04-25T11:38:26.673702+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:38:26.673702+0000","last_undegraded":"2024-04-25T11:38:26.673702+0000","last_fullsized":"2024-04-25T11:38:26.673702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674250+0000","last_change":"2024-04-25T11:38:25.680252+0000","last_active":"2024-04-25T11:38:26.674250+0000","last_peered":"2024-04-25T11:38:26.674250+0000","last_clean":"2024-04-25T11:38:26.674250+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:38:26.674250+0000","last_undegraded":"2024-04-25T11:38:26.674250+0000","last_fullsized":"2024-04-25T11:38:26.674250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762291+0000","last_change":"2024-04-25T11:38:25.669542+0000","last_active":"2024-04-25T11:38:26.762291+0000","last_peered":"2024-04-25T11:38:26.762291+0000","last_clean":"2024-04-25T11:38:26.762291+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:38:26.762291+0000","last_undegraded":"2024-04-25T11:38:26.762291+0000","last_fullsized":"2024-04-25T11:38:26.762291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:51.929330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.605314+0000","last_change":"2024-04-25T11:38:24.666197+0000","last_active":"2024-04-25T11:39:01.605314+0000","last_peered":"2024-04-25T11:39:01.605314+0000","last_clean":"2024-04-25T11:39:01.605314+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:39:01.605314+0000","last_undegraded":"2024-04-25T11:39:01.605314+0000","last_fullsized":"2024-04-25T11:39:01.605314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.198972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761974+0000","last_change":"2024-04-25T11:38:25.682931+0000","last_active":"2024-04-25T11:38:26.761974+0000","last_peered":"2024-04-25T11:38:26.761974+0000","last_clean":"2024-04-25T11:38:26.761974+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:38:26.761974+0000","last_undegraded":"2024-04-25T11:38:26.761974+0000","last_fullsized":"2024-04-25T11:38:26.761974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028972+0000","last_change":"2024-04-25T11:38:24.660888+0000","last_active":"2024-04-25T11:38:47.028972+0000","last_peered":"2024-04-25T11:38:47.028972+0000","last_clean":"2024-04-25T11:38:47.028972+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:38:47.028972+0000","last_undegraded":"2024-04-25T11:38:47.028972+0000","last_fullsized":"2024-04-25T11:38:47.028972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:41.729888+0000","last_change":"2024-04-25T11:38:24.657815+0000","last_active":"2024-04-25T11:38:41.729888+0000","last_peered":"2024-04-25T11:38:41.729888+0000","last_clean":"2024-04-25T11:38:41.729888+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:38:41.729888+0000","last_undegraded":"2024-04-25T11:38:41.729888+0000","last_fullsized":"2024-04-25T11:38:41.729888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761437+0000","last_change":"2024-04-25T11:38:25.680969+0000","last_active":"2024-04-25T11:38:26.761437+0000","last_peered":"2024-04-25T11:38:26.761437+0000","last_clean":"2024-04-25T11:38:26.761437+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:38:26.761437+0000","last_undegraded":"2024-04-25T11:38:26.761437+0000","last_fullsized":"2024-04-25T11:38:26.761437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028798+0000","last_change":"2024-04-25T11:38:24.668899+0000","last_active":"2024-04-25T11:38:47.028798+0000","last_peered":"2024-04-25T11:38:47.028798+0000","last_clean":"2024-04-25T11:38:47.028798+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:38:47.028798+0000","last_undegraded":"2024-04-25T11:38:47.028798+0000","last_fullsized":"2024-04-25T11:38:47.028798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672975+0000","last_change":"2024-04-25T11:38:25.666511+0000","last_active":"2024-04-25T11:38:26.672975+0000","last_peered":"2024-04-25T11:38:26.672975+0000","last_clean":"2024-04-25T11:38:26.672975+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:38:26.672975+0000","last_undegraded":"2024-04-25T11:38:26.672975+0000","last_fullsized":"2024-04-25T11:38:26.672975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.605159+0000","last_change":"2024-04-25T11:38:24.666735+0000","last_active":"2024-04-25T11:39:01.605159+0000","last_peered":"2024-04-25T11:39:01.605159+0000","last_clean":"2024-04-25T11:39:01.605159+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:39:01.605159+0000","last_undegraded":"2024-04-25T11:39:01.605159+0000","last_fullsized":"2024-04-25T11:39:01.605159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763586+0000","last_change":"2024-04-25T11:38:25.680065+0000","last_active":"2024-04-25T11:38:26.763586+0000","last_peered":"2024-04-25T11:38:26.763586+0000","last_clean":"2024-04-25T11:38:26.763586+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:38:26.763586+0000","last_undegraded":"2024-04-25T11:38:26.763586+0000","last_fullsized":"2024-04-25T11:38:26.763586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.048969+0000","last_change":"2024-04-25T11:38:24.671566+0000","last_active":"2024-04-25T11:38:53.048969+0000","last_peered":"2024-04-25T11:38:53.048969+0000","last_clean":"2024-04-25T11:38:53.048969+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:38:53.048969+0000","last_undegraded":"2024-04-25T11:38:53.048969+0000","last_fullsized":"2024-04-25T11:38:53.048969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761910+0000","last_change":"2024-04-25T11:38:25.678075+0000","last_active":"2024-04-25T11:38:26.761910+0000","last_peered":"2024-04-25T11:38:26.761910+0000","last_clean":"2024-04-25T11:38:26.761910+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:38:26.761910+0000","last_undegraded":"2024-04-25T11:38:26.761910+0000","last_fullsized":"2024-04-25T11:38:26.761910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:35.703616+0000","last_change":"2024-04-25T11:38:24.667282+0000","last_active":"2024-04-25T11:38:35.703616+0000","last_peered":"2024-04-25T11:38:35.703616+0000","last_clean":"2024-04-25T11:38:35.703616+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:38:35.703616+0000","last_undegraded":"2024-04-25T11:38:35.703616+0000","last_fullsized":"2024-04-25T11:38:35.703616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762833+0000","last_change":"2024-04-25T11:38:25.680417+0000","last_active":"2024-04-25T11:38:26.762833+0000","last_peered":"2024-04-25T11:38:26.762833+0000","last_clean":"2024-04-25T11:38:26.762833+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:38:26.762833+0000","last_undegraded":"2024-04-25T11:38:26.762833+0000","last_fullsized":"2024-04-25T11:38:26.762833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'14","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:05.634385+0000","last_change":"2024-04-25T11:38:24.662562+0000","last_active":"2024-04-25T11:39:05.634385+0000","last_peered":"2024-04-25T11:39:05.634385+0000","last_clean":"2024-04-25T11:39:05.634385+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:39:05.634385+0000","last_undegraded":"2024-04-25T11:39:05.634385+0000","last_fullsized":"2024-04-25T11:39:05.634385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762135+0000","last_change":"2024-04-25T11:38:25.677518+0000","last_active":"2024-04-25T11:38:26.762135+0000","last_peered":"2024-04-25T11:38:26.762135+0000","last_clean":"2024-04-25T11:38:26.762135+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:38:26.762135+0000","last_undegraded":"2024-04-25T11:38:26.762135+0000","last_fullsized":"2024-04-25T11:38:26.762135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.053184+0000","last_change":"2024-04-25T11:38:24.658960+0000","last_active":"2024-04-25T11:38:53.053184+0000","last_peered":"2024-04-25T11:38:53.053184+0000","last_clean":"2024-04-25T11:38:53.053184+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:38:53.053184+0000","last_undegraded":"2024-04-25T11:38:53.053184+0000","last_fullsized":"2024-04-25T11:38:53.053184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761438+0000","last_change":"2024-04-25T11:38:25.681355+0000","last_active":"2024-04-25T11:38:26.761438+0000","last_peered":"2024-04-25T11:38:26.761438+0000","last_clean":"2024-04-25T11:38:26.761438+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:38:26.761438+0000","last_undegraded":"2024-04-25T11:38:26.761438+0000","last_fullsized":"2024-04-25T11:38:26.761438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.029219+0000","last_change":"2024-04-25T11:38:24.666747+0000","last_active":"2024-04-25T11:38:47.029219+0000","last_peered":"2024-04-25T11:38:47.029219+0000","last_clean":"2024-04-25T11:38:47.029219+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:38:47.029219+0000","last_undegraded":"2024-04-25T11:38:47.029219+0000","last_fullsized":"2024-04-25T11:38:47.029219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762221+0000","last_change":"2024-04-25T11:38:25.684128+0000","last_active":"2024-04-25T11:38:26.762221+0000","last_peered":"2024-04-25T11:38:26.762221+0000","last_clean":"2024-04-25T11:38:26.762221+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:38:26.762221+0000","last_undegraded":"2024-04-25T11:38:26.762221+0000","last_fullsized":"2024-04-25T11:38:26.762221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.607906+0000","last_change":"2024-04-25T11:38:24.661103+0000","last_active":"2024-04-25T11:39:01.607906+0000","last_peered":"2024-04-25T11:39:01.607906+0000","last_clean":"2024-04-25T11:39:01.607906+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:39:01.607906+0000","last_undegraded":"2024-04-25T11:39:01.607906+0000","last_fullsized":"2024-04-25T11:39:01.607906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674380+0000","last_change":"2024-04-25T11:38:25.680490+0000","last_active":"2024-04-25T11:38:26.674380+0000","last_peered":"2024-04-25T11:38:26.674380+0000","last_clean":"2024-04-25T11:38:26.674380+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:38:26.674380+0000","last_undegraded":"2024-04-25T11:38:26.674380+0000","last_fullsized":"2024-04-25T11:38:26.674380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.024193+0000","last_change":"2024-04-25T11:38:24.665238+0000","last_active":"2024-04-25T11:38:47.024193+0000","last_peered":"2024-04-25T11:38:47.024193+0000","last_clean":"2024-04-25T11:38:47.024193+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:38:47.024193+0000","last_undegraded":"2024-04-25T11:38:47.024193+0000","last_fullsized":"2024-04-25T11:38:47.024193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761773+0000","last_change":"2024-04-25T11:38:25.682669+0000","last_active":"2024-04-25T11:38:26.761773+0000","last_peered":"2024-04-25T11:38:26.761773+0000","last_clean":"2024-04-25T11:38:26.761773+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:38:26.761773+0000","last_undegraded":"2024-04-25T11:38:26.761773+0000","last_fullsized":"2024-04-25T11:38:26.761773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679038+0000","last_change":"2024-04-25T11:38:24.658666+0000","last_active":"2024-04-25T11:38:30.679038+0000","last_peered":"2024-04-25T11:38:30.679038+0000","last_clean":"2024-04-25T11:38:30.679038+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:38:30.679038+0000","last_undegraded":"2024-04-25T11:38:30.679038+0000","last_fullsized":"2024-04-25T11:38:30.679038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:24.295273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762494+0000","last_change":"2024-04-25T11:38:25.680783+0000","last_active":"2024-04-25T11:38:26.762494+0000","last_peered":"2024-04-25T11:38:26.762494+0000","last_clean":"2024-04-25T11:38:26.762494+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:38:26.762494+0000","last_undegraded":"2024-04-25T11:38:26.762494+0000","last_fullsized":"2024-04-25T11:38:26.762494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.048152+0000","last_change":"2024-04-25T11:38:24.656937+0000","last_active":"2024-04-25T11:38:53.048152+0000","last_peered":"2024-04-25T11:38:53.048152+0000","last_clean":"2024-04-25T11:38:53.048152+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:38:53.048152+0000","last_undegraded":"2024-04-25T11:38:53.048152+0000","last_fullsized":"2024-04-25T11:38:53.048152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763039+0000","last_change":"2024-04-25T11:38:25.681307+0000","last_active":"2024-04-25T11:38:26.763039+0000","last_peered":"2024-04-25T11:38:26.763039+0000","last_clean":"2024-04-25T11:38:26.763039+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:38:26.763039+0000","last_undegraded":"2024-04-25T11:38:26.763039+0000","last_fullsized":"2024-04-25T11:38:26.763039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:17.678183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028781+0000","last_change":"2024-04-25T11:38:24.663247+0000","last_active":"2024-04-25T11:38:47.028781+0000","last_peered":"2024-04-25T11:38:47.028781+0000","last_clean":"2024-04-25T11:38:47.028781+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:38:47.028781+0000","last_undegraded":"2024-04-25T11:38:47.028781+0000","last_fullsized":"2024-04-25T11:38:47.028781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763176+0000","last_change":"2024-04-25T11:38:25.676409+0000","last_active":"2024-04-25T11:38:26.763176+0000","last_peered":"2024-04-25T11:38:26.763176+0000","last_clean":"2024-04-25T11:38:26.763176+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.763176+0000","last_undegraded":"2024-04-25T11:38:26.763176+0000","last_fullsized":"2024-04-25T11:38:26.763176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.602338+0000","last_change":"2024-04-25T11:38:24.666800+0000","last_active":"2024-04-25T11:39:01.602338+0000","last_peered":"2024-04-25T11:39:01.602338+0000","last_clean":"2024-04-25T11:39:01.602338+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:39:01.602338+0000","last_undegraded":"2024-04-25T11:39:01.602338+0000","last_fullsized":"2024-04-25T11:39:01.602338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762286+0000","last_change":"2024-04-25T11:38:25.678136+0000","last_active":"2024-04-25T11:38:26.762286+0000","last_peered":"2024-04-25T11:38:26.762286+0000","last_clean":"2024-04-25T11:38:26.762286+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:38:26.762286+0000","last_undegraded":"2024-04-25T11:38:26.762286+0000","last_fullsized":"2024-04-25T11:38:26.762286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761547+0000","last_change":"2024-04-25T11:38:20.865911+0000","last_active":"2024-04-25T11:38:26.761547+0000","last_peered":"2024-04-25T11:38:26.761547+0000","last_clean":"2024-04-25T11:38:26.761547+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:38:26.761547+0000","last_undegraded":"2024-04-25T11:38:26.761547+0000","last_fullsized":"2024-04-25T11:38:26.761547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_clean_scrub_stamp":"2024-04-25T11:38:19.606342+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:58.126842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.026102+0000","last_change":"2024-04-25T11:38:24.671283+0000","last_active":"2024-04-25T11:38:47.026102+0000","last_peered":"2024-04-25T11:38:47.026102+0000","last_clean":"2024-04-25T11:38:47.026102+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:38:47.026102+0000","last_undegraded":"2024-04-25T11:38:47.026102+0000","last_fullsized":"2024-04-25T11:38:47.026102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761567+0000","last_change":"2024-04-25T11:38:25.675002+0000","last_active":"2024-04-25T11:38:26.761567+0000","last_peered":"2024-04-25T11:38:26.761567+0000","last_clean":"2024-04-25T11:38:26.761567+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:38:26.761567+0000","last_undegraded":"2024-04-25T11:38:26.761567+0000","last_fullsized":"2024-04-25T11:38:26.761567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:41.726797+0000","last_change":"2024-04-25T11:38:24.671213+0000","last_active":"2024-04-25T11:38:41.726797+0000","last_peered":"2024-04-25T11:38:41.726797+0000","last_clean":"2024-04-25T11:38:41.726797+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:38:41.726797+0000","last_undegraded":"2024-04-25T11:38:41.726797+0000","last_fullsized":"2024-04-25T11:38:41.726797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761052+0000","last_change":"2024-04-25T11:38:25.673353+0000","last_active":"2024-04-25T11:38:26.761052+0000","last_peered":"2024-04-25T11:38:26.761052+0000","last_clean":"2024-04-25T11:38:26.761052+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:38:26.761052+0000","last_undegraded":"2024-04-25T11:38:26.761052+0000","last_fullsized":"2024-04-25T11:38:26.761052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:48.553601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.052041+0000","last_change":"2024-04-25T11:38:24.650599+0000","last_active":"2024-04-25T11:38:53.052041+0000","last_peered":"2024-04-25T11:38:53.052041+0000","last_clean":"2024-04-25T11:38:53.052041+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:38:53.052041+0000","last_undegraded":"2024-04-25T11:38:53.052041+0000","last_fullsized":"2024-04-25T11:38:53.052041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672982+0000","last_change":"2024-04-25T11:38:25.668956+0000","last_active":"2024-04-25T11:38:26.672982+0000","last_peered":"2024-04-25T11:38:26.672982+0000","last_clean":"2024-04-25T11:38:26.672982+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:38:26.672982+0000","last_undegraded":"2024-04-25T11:38:26.672982+0000","last_fullsized":"2024-04-25T11:38:26.672982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.053634+0000","last_change":"2024-04-25T11:38:24.665496+0000","last_active":"2024-04-25T11:38:53.053634+0000","last_peered":"2024-04-25T11:38:53.053634+0000","last_clean":"2024-04-25T11:38:53.053634+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:38:53.053634+0000","last_undegraded":"2024-04-25T11:38:53.053634+0000","last_fullsized":"2024-04-25T11:38:53.053634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762812+0000","last_change":"2024-04-25T11:38:25.681800+0000","last_active":"2024-04-25T11:38:26.762812+0000","last_peered":"2024-04-25T11:38:26.762812+0000","last_clean":"2024-04-25T11:38:26.762812+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:38:26.762812+0000","last_undegraded":"2024-04-25T11:38:26.762812+0000","last_fullsized":"2024-04-25T11:38:26.762812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:39.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.589274+0000","last_change":"2024-04-25T11:38:24.659792+0000","last_active":"2024-04-25T11:38:28.589274+0000","last_peered":"2024-04-25T11:38:28.589274+0000","last_clean":"2024-04-25T11:38:28.589274+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:38:28.589274+0000","last_undegraded":"2024-04-25T11:38:28.589274+0000","last_fullsized":"2024-04-25T11:38:28.589274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763580+0000","last_change":"2024-04-25T11:38:25.673411+0000","last_active":"2024-04-25T11:38:26.763580+0000","last_peered":"2024-04-25T11:38:26.763580+0000","last_clean":"2024-04-25T11:38:26.763580+0000","last_became_active":"2024-04-25T11:38:25.673116+0000","last_became_peered":"2024-04-25T11:38:25.673116+0000","last_unstale":"2024-04-25T11:38:26.763580+0000","last_undegraded":"2024-04-25T11:38:26.763580+0000","last_fullsized":"2024-04-25T11:38:26.763580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.024457+0000","last_change":"2024-04-25T11:38:24.671412+0000","last_active":"2024-04-25T11:38:47.024457+0000","last_peered":"2024-04-25T11:38:47.024457+0000","last_clean":"2024-04-25T11:38:47.024457+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:38:47.024457+0000","last_undegraded":"2024-04-25T11:38:47.024457+0000","last_fullsized":"2024-04-25T11:38:47.024457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:34.607841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673048+0000","last_change":"2024-04-25T11:38:25.668959+0000","last_active":"2024-04-25T11:38:26.673048+0000","last_peered":"2024-04-25T11:38:26.673048+0000","last_clean":"2024-04-25T11:38:26.673048+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:38:26.673048+0000","last_undegraded":"2024-04-25T11:38:26.673048+0000","last_fullsized":"2024-04-25T11:38:26.673048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028863+0000","last_change":"2024-04-25T11:38:24.664146+0000","last_active":"2024-04-25T11:38:47.028863+0000","last_peered":"2024-04-25T11:38:47.028863+0000","last_clean":"2024-04-25T11:38:47.028863+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:38:47.028863+0000","last_undegraded":"2024-04-25T11:38:47.028863+0000","last_fullsized":"2024-04-25T11:38:47.028863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761339+0000","last_change":"2024-04-25T11:38:25.680105+0000","last_active":"2024-04-25T11:38:26.761339+0000","last_peered":"2024-04-25T11:38:26.761339+0000","last_clean":"2024-04-25T11:38:26.761339+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:38:26.761339+0000","last_undegraded":"2024-04-25T11:38:26.761339+0000","last_fullsized":"2024-04-25T11:38:26.761339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.049396+0000","last_change":"2024-04-25T11:38:24.668369+0000","last_active":"2024-04-25T11:38:53.049396+0000","last_peered":"2024-04-25T11:38:53.049396+0000","last_clean":"2024-04-25T11:38:53.049396+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:38:53.049396+0000","last_undegraded":"2024-04-25T11:38:53.049396+0000","last_fullsized":"2024-04-25T11:38:53.049396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761723+0000","last_change":"2024-04-25T11:38:25.676446+0000","last_active":"2024-04-25T11:38:26.761723+0000","last_peered":"2024-04-25T11:38:26.761723+0000","last_clean":"2024-04-25T11:38:26.761723+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.761723+0000","last_undegraded":"2024-04-25T11:38:26.761723+0000","last_fullsized":"2024-04-25T11:38:26.761723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:35.700179+0000","last_change":"2024-04-25T11:38:24.671951+0000","last_active":"2024-04-25T11:38:35.700179+0000","last_peered":"2024-04-25T11:38:35.700179+0000","last_clean":"2024-04-25T11:38:35.700179+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:38:35.700179+0000","last_undegraded":"2024-04-25T11:38:35.700179+0000","last_fullsized":"2024-04-25T11:38:35.700179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761852+0000","last_change":"2024-04-25T11:38:25.669474+0000","last_active":"2024-04-25T11:38:26.761852+0000","last_peered":"2024-04-25T11:38:26.761852+0000","last_clean":"2024-04-25T11:38:26.761852+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:38:26.761852+0000","last_undegraded":"2024-04-25T11:38:26.761852+0000","last_fullsized":"2024-04-25T11:38:26.761852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:38.359208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.048585+0000","last_change":"2024-04-25T11:38:24.668689+0000","last_active":"2024-04-25T11:38:53.048585+0000","last_peered":"2024-04-25T11:38:53.048585+0000","last_clean":"2024-04-25T11:38:53.048585+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:38:53.048585+0000","last_undegraded":"2024-04-25T11:38:53.048585+0000","last_fullsized":"2024-04-25T11:38:53.048585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761421+0000","last_change":"2024-04-25T11:38:25.668088+0000","last_active":"2024-04-25T11:38:26.761421+0000","last_peered":"2024-04-25T11:38:26.761421+0000","last_clean":"2024-04-25T11:38:26.761421+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:38:26.761421+0000","last_undegraded":"2024-04-25T11:38:26.761421+0000","last_fullsized":"2024-04-25T11:38:26.761421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.029132+0000","last_change":"2024-04-25T11:38:24.662913+0000","last_active":"2024-04-25T11:38:47.029132+0000","last_peered":"2024-04-25T11:38:47.029132+0000","last_clean":"2024-04-25T11:38:47.029132+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:38:47.029132+0000","last_undegraded":"2024-04-25T11:38:47.029132+0000","last_fullsized":"2024-04-25T11:38:47.029132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761308+0000","last_change":"2024-04-25T11:38:25.680244+0000","last_active":"2024-04-25T11:38:26.761308+0000","last_peered":"2024-04-25T11:38:26.761308+0000","last_clean":"2024-04-25T11:38:26.761308+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:38:26.761308+0000","last_undegraded":"2024-04-25T11:38:26.761308+0000","last_fullsized":"2024-04-25T11:38:26.761308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:35.702098+0000","last_change":"2024-04-25T11:38:24.669196+0000","last_active":"2024-04-25T11:38:35.702098+0000","last_peered":"2024-04-25T11:38:35.702098+0000","last_clean":"2024-04-25T11:38:35.702098+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:38:35.702098+0000","last_undegraded":"2024-04-25T11:38:35.702098+0000","last_fullsized":"2024-04-25T11:38:35.702098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.791358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674198+0000","last_change":"2024-04-25T11:38:25.668400+0000","last_active":"2024-04-25T11:38:26.674198+0000","last_peered":"2024-04-25T11:38:26.674198+0000","last_clean":"2024-04-25T11:38:26.674198+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:38:26.674198+0000","last_undegraded":"2024-04-25T11:38:26.674198+0000","last_fullsized":"2024-04-25T11:38:26.674198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763520+0000","last_change":"2024-04-25T11:38:25.683663+0000","last_active":"2024-04-25T11:38:26.763520+0000","last_peered":"2024-04-25T11:38:26.763520+0000","last_clean":"2024-04-25T11:38:26.763520+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:38:26.763520+0000","last_undegraded":"2024-04-25T11:38:26.763520+0000","last_fullsized":"2024-04-25T11:38:26.763520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.049503+0000","last_change":"2024-04-25T11:38:24.670671+0000","last_active":"2024-04-25T11:38:53.049503+0000","last_peered":"2024-04-25T11:38:53.049503+0000","last_clean":"2024-04-25T11:38:53.049503+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:38:53.049503+0000","last_undegraded":"2024-04-25T11:38:53.049503+0000","last_fullsized":"2024-04-25T11:38:53.049503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763017+0000","last_change":"2024-04-25T11:38:25.681439+0000","last_active":"2024-04-25T11:38:26.763017+0000","last_peered":"2024-04-25T11:38:26.763017+0000","last_clean":"2024-04-25T11:38:26.763017+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:38:26.763017+0000","last_undegraded":"2024-04-25T11:38:26.763017+0000","last_fullsized":"2024-04-25T11:38:26.763017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.050209+0000","last_change":"2024-04-25T11:38:24.669485+0000","last_active":"2024-04-25T11:38:53.050209+0000","last_peered":"2024-04-25T11:38:53.050209+0000","last_clean":"2024-04-25T11:38:53.050209+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:38:53.050209+0000","last_undegraded":"2024-04-25T11:38:53.050209+0000","last_fullsized":"2024-04-25T11:38:53.050209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761766+0000","last_change":"2024-04-25T11:38:25.684415+0000","last_active":"2024-04-25T11:38:26.761766+0000","last_peered":"2024-04-25T11:38:26.761766+0000","last_clean":"2024-04-25T11:38:26.761766+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:38:26.761766+0000","last_undegraded":"2024-04-25T11:38:26.761766+0000","last_fullsized":"2024-04-25T11:38:26.761766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.678908+0000","last_change":"2024-04-25T11:38:24.658741+0000","last_active":"2024-04-25T11:38:30.678908+0000","last_peered":"2024-04-25T11:38:30.678908+0000","last_clean":"2024-04-25T11:38:30.678908+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:38:30.678908+0000","last_undegraded":"2024-04-25T11:38:30.678908+0000","last_fullsized":"2024-04-25T11:38:30.678908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761168+0000","last_change":"2024-04-25T11:38:25.667505+0000","last_active":"2024-04-25T11:38:26.761168+0000","last_peered":"2024-04-25T11:38:26.761168+0000","last_clean":"2024-04-25T11:38:26.761168+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:38:26.761168+0000","last_undegraded":"2024-04-25T11:38:26.761168+0000","last_fullsized":"2024-04-25T11:38:26.761168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.049060+0000","last_change":"2024-04-25T11:38:24.669215+0000","last_active":"2024-04-25T11:38:53.049060+0000","last_peered":"2024-04-25T11:38:53.049060+0000","last_clean":"2024-04-25T11:38:53.049060+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:38:53.049060+0000","last_undegraded":"2024-04-25T11:38:53.049060+0000","last_fullsized":"2024-04-25T11:38:53.049060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11177,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":218,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":22354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":663984,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":75315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":75448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":75349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":662176,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:39:12.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:39:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.471+0000 7f20522cf640 1 Processor -- start 2024-04-25T11:39:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.471+0000 7f20522cf640 1 -- start start 2024-04-25T11:39:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f20522cf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f204c155010 0x7f204c0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f20522cf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f204c1553f0 con 0x7f204c06ca80 2024-04-25T11:39:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f20522cf640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f204c0ce7c0 con 0x7f204c0c97f0 2024-04-25T11:39:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f20522cf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f204c06eab0 con 0x7f204c155010 2024-04-25T11:39:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f204bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f204c155010 0x7f204c0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f205086c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f204c06ca80 0x7f204c165570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52738/0 (socket says 172.21.15.40:52738) 2024-04-25T11:39:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f205086c640 1 -- 172.21.15.40:0/3685746889 learned_addr learned my addr 172.21.15.40:0/3685746889 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f204bfff640 1 -- 172.21.15.40:0/3685746889 >> v1:172.21.15.62:6789/0 conn(0x7f204c0c97f0 legacy=0x7f204c15eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f204bfff640 1 -- 172.21.15.40:0/3685746889 >> v1:172.21.15.40:6789/0 conn(0x7f204c06ca80 legacy=0x7f204c165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f204affd640 1 -- 172.21.15.40:0/3685746889 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1946796515 0 0) 0x7f204c1553f0 con 0x7f204c06ca80 2024-04-25T11:39:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.472+0000 7f204bfff640 1 -- 172.21.15.40:0/3685746889 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c0ceaa0 con 0x7f204c155010 2024-04-25T11:39:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.473+0000 7f204bfff640 1 --2- 172.21.15.40:0/3685746889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c155010 0x7f204c0c8c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f203c0097a0 tx=0x7f203c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dfaac8130e08a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.473+0000 7f204affd640 1 -- 172.21.15.40:0/3685746889 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c037070 con 0x7f204c155010 2024-04-25T11:39:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.473+0000 7f204affd640 1 -- 172.21.15.40:0/3685746889 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203c032440 con 0x7f204c155010 2024-04-25T11:39:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.473+0000 7f204affd640 1 -- 172.21.15.40:0/3685746889 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c032810 con 0x7f204c155010 2024-04-25T11:39:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.474+0000 7f20522cf640 1 -- 172.21.15.40:0/3685746889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c155010 msgr2=0x7f204c0c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.474+0000 7f20522cf640 1 --2- 172.21.15.40:0/3685746889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c155010 0x7f204c0c8c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f203c0097a0 tx=0x7f203c02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.474+0000 7f20522cf640 1 -- 172.21.15.40:0/3685746889 shutdown_connections 2024-04-25T11:39:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.474+0000 7f20522cf640 1 --2- 172.21.15.40:0/3685746889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c155010 0x7f204c0c8c50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.474+0000 7f20522cf640 1 -- 172.21.15.40:0/3685746889 >> 172.21.15.40:0/3685746889 conn(0x7f204c06b970 msgr2=0x7f204c06f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.474+0000 7f20522cf640 1 -- 172.21.15.40:0/3685746889 shutdown_connections 2024-04-25T11:39:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.474+0000 7f20522cf640 1 -- 172.21.15.40:0/3685746889 wait complete. 2024-04-25T11:39:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.475+0000 7f20522cf640 1 Processor -- start 2024-04-25T11:39:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.475+0000 7f20522cf640 1 -- start start 2024-04-25T11:39:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f20522cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06ca80 0x7f204c152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f20522cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0c97f0 0x7f204c1e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f20522cf640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f204c155010 0x7f204c0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f20522cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f204c0ced80 con 0x7f204c0c97f0 2024-04-25T11:39:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f20522cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f204c0c9300 con 0x7f204c06ca80 2024-04-25T11:39:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f20522cf640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f204c15e5f0 con 0x7f204c155010 2024-04-25T11:39:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0c97f0 0x7f204c1e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0c97f0 0x7f204c1e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45622/0 (socket says 172.21.15.40:45622) 2024-04-25T11:39:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06ca80 0x7f204c152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204b7fe640 1 -- 172.21.15.40:0/2798061924 learned_addr learned my addr 172.21.15.40:0/2798061924 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f205086c640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f204c155010 0x7f204c0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204b7fe640 1 -- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f204c155010 msgr2=0x7f204c0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204b7fe640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f204c155010 0x7f204c0cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204b7fe640 1 -- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06ca80 msgr2=0x7f204c152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204b7fe640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06ca80 0x7f204c152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204b7fe640 1 -- 172.21.15.40:0/2798061924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c0caf40 con 0x7f204c0c97f0 2024-04-25T11:39:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.476+0000 7f204b7fe640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0c97f0 0x7f204c1e4580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f204000c400 tx=0x7f204000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.477+0000 7f2048ff9640 1 -- 172.21.15.40:0/2798061924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040017070 con 0x7f204c0c97f0 2024-04-25T11:39:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.477+0000 7f2048ff9640 1 -- 172.21.15.40:0/2798061924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20400052d0 con 0x7f204c0c97f0 2024-04-25T11:39:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.477+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c1e5120 con 0x7f204c0c97f0 2024-04-25T11:39:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.477+0000 7f2048ff9640 1 -- 172.21.15.40:0/2798061924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040003830 con 0x7f204c0c97f0 2024-04-25T11:39:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.477+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c0cb810 con 0x7f204c0c97f0 2024-04-25T11:39:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.478+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f204c153590 con 0x7f204c0c97f0 2024-04-25T11:39:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.478+0000 7f2048ff9640 1 -- 172.21.15.40:0/2798061924 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f204001a070 con 0x7f204c0c97f0 2024-04-25T11:39:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.479+0000 7f2048ff9640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f203806c8f0 0x7f203806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.479+0000 7f2048ff9640 1 -- 172.21.15.40:0/2798061924 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f204008ad80 con 0x7f204c0c97f0 2024-04-25T11:39:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.479+0000 7f204bfff640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f203806c8f0 0x7f203806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.479+0000 7f204bfff640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f203806c8f0 0x7f203806edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f203c034000 tx=0x7f203c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.483+0000 7f2048ff9640 1 -- 172.21.15.40:0/2798061924 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f204c153590 con 0x7f204c0c97f0 2024-04-25T11:39:12.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.665+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f204c068bb0 con 0x7f203806c8f0 2024-04-25T11:39:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.673+0000 7f2048ff9640 1 -- 172.21.15.40:0/2798061924 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252064 (secure 0 0 0) 0x7f204c068bb0 con 0x7f203806c8f0 2024-04-25T11:39:12.663 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:12.664 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:39:12.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f203806c8f0 msgr2=0x7f203806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f203806c8f0 0x7f203806edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f203c034000 tx=0x7f203c02e000 comp rx=0 tx=0).stop 2024-04-25T11:39:12.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0c97f0 msgr2=0x7f204c1e4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0c97f0 0x7f204c1e4580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f204000c400 tx=0x7f204000c8c0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 shutdown_connections 2024-04-25T11:39:12.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f203806c8f0 0x7f203806edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f204c155010 0x7f204c0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0c97f0 0x7f204c1e4580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 --2- 172.21.15.40:0/2798061924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06ca80 0x7f204c152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 >> 172.21.15.40:0/2798061924 conn(0x7f204c06b970 msgr2=0x7f204c0c3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:12.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 shutdown_connections 2024-04-25T11:39:12.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.676+0000 7f20522cf640 1 -- 172.21.15.40:0/2798061924 wait complete. 2024-04-25T11:39:12.674 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-25T11:39:10.959552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601545,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":377,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218112,"kb_used_data":1984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":2031616,"data_stored":1770562,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3143,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002458"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762879+0000","last_change":"2024-04-25T11:38:25.677572+0000","last_active":"2024-04-25T11:38:26.762879+0000","last_peered":"2024-04-25T11:38:26.762879+0000","last_clean":"2024-04-25T11:38:26.762879+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:38:26.762879+0000","last_undegraded":"2024-04-25T11:38:26.762879+0000","last_fullsized":"2024-04-25T11:38:26.762879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673367+0000","last_change":"2024-04-25T11:38:25.677437+0000","last_active":"2024-04-25T11:38:26.673367+0000","last_peered":"2024-04-25T11:38:26.673367+0000","last_clean":"2024-04-25T11:38:26.673367+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:38:26.673367+0000","last_undegraded":"2024-04-25T11:38:26.673367+0000","last_fullsized":"2024-04-25T11:38:26.673367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761847+0000","last_change":"2024-04-25T11:38:25.676553+0000","last_active":"2024-04-25T11:38:26.761847+0000","last_peered":"2024-04-25T11:38:26.761847+0000","last_clean":"2024-04-25T11:38:26.761847+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:38:26.761847+0000","last_undegraded":"2024-04-25T11:38:26.761847+0000","last_fullsized":"2024-04-25T11:38:26.761847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762862+0000","last_change":"2024-04-25T11:38:25.676518+0000","last_active":"2024-04-25T11:38:26.762862+0000","last_peered":"2024-04-25T11:38:26.762862+0000","last_clean":"2024-04-25T11:38:26.762862+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:38:26.762862+0000","last_undegraded":"2024-04-25T11:38:26.762862+0000","last_fullsized":"2024-04-25T11:38:26.762862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762062+0000","last_change":"2024-04-25T11:38:25.678524+0000","last_active":"2024-04-25T11:38:26.762062+0000","last_peered":"2024-04-25T11:38:26.762062+0000","last_clean":"2024-04-25T11:38:26.762062+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:38:26.762062+0000","last_undegraded":"2024-04-25T11:38:26.762062+0000","last_fullsized":"2024-04-25T11:38:26.762062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761774+0000","last_change":"2024-04-25T11:38:25.683983+0000","last_active":"2024-04-25T11:38:26.761774+0000","last_peered":"2024-04-25T11:38:26.761774+0000","last_clean":"2024-04-25T11:38:26.761774+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:38:26.761774+0000","last_undegraded":"2024-04-25T11:38:26.761774+0000","last_fullsized":"2024-04-25T11:38:26.761774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761720+0000","last_change":"2024-04-25T11:38:25.682904+0000","last_active":"2024-04-25T11:38:26.761720+0000","last_peered":"2024-04-25T11:38:26.761720+0000","last_clean":"2024-04-25T11:38:26.761720+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:38:26.761720+0000","last_undegraded":"2024-04-25T11:38:26.761720+0000","last_fullsized":"2024-04-25T11:38:26.761720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761274+0000","last_change":"2024-04-25T11:38:25.670312+0000","last_active":"2024-04-25T11:38:26.761274+0000","last_peered":"2024-04-25T11:38:26.761274+0000","last_clean":"2024-04-25T11:38:26.761274+0000","last_became_active":"2024-04-25T11:38:25.669988+0000","last_became_peered":"2024-04-25T11:38:25.669988+0000","last_unstale":"2024-04-25T11:38:26.761274+0000","last_undegraded":"2024-04-25T11:38:26.761274+0000","last_fullsized":"2024-04-25T11:38:26.761274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762939+0000","last_change":"2024-04-25T11:38:25.678459+0000","last_active":"2024-04-25T11:38:26.762939+0000","last_peered":"2024-04-25T11:38:26.762939+0000","last_clean":"2024-04-25T11:38:26.762939+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:38:26.762939+0000","last_undegraded":"2024-04-25T11:38:26.762939+0000","last_fullsized":"2024-04-25T11:38:26.762939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674629+0000","last_change":"2024-04-25T11:38:25.675561+0000","last_active":"2024-04-25T11:38:26.674629+0000","last_peered":"2024-04-25T11:38:26.674629+0000","last_clean":"2024-04-25T11:38:26.674629+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:38:26.674629+0000","last_undegraded":"2024-04-25T11:38:26.674629+0000","last_fullsized":"2024-04-25T11:38:26.674629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674251+0000","last_change":"2024-04-25T11:38:25.671113+0000","last_active":"2024-04-25T11:38:26.674251+0000","last_peered":"2024-04-25T11:38:26.674251+0000","last_clean":"2024-04-25T11:38:26.674251+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:38:26.674251+0000","last_undegraded":"2024-04-25T11:38:26.674251+0000","last_fullsized":"2024-04-25T11:38:26.674251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762821+0000","last_change":"2024-04-25T11:38:25.665690+0000","last_active":"2024-04-25T11:38:26.762821+0000","last_peered":"2024-04-25T11:38:26.762821+0000","last_clean":"2024-04-25T11:38:26.762821+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:38:26.762821+0000","last_undegraded":"2024-04-25T11:38:26.762821+0000","last_fullsized":"2024-04-25T11:38:26.762821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:25.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761152+0000","last_change":"2024-04-25T11:38:25.666858+0000","last_active":"2024-04-25T11:38:26.761152+0000","last_peered":"2024-04-25T11:38:26.761152+0000","last_clean":"2024-04-25T11:38:26.761152+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:38:26.761152+0000","last_undegraded":"2024-04-25T11:38:26.761152+0000","last_fullsized":"2024-04-25T11:38:26.761152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761293+0000","last_change":"2024-04-25T11:38:25.677561+0000","last_active":"2024-04-25T11:38:26.761293+0000","last_peered":"2024-04-25T11:38:26.761293+0000","last_clean":"2024-04-25T11:38:26.761293+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:38:26.761293+0000","last_undegraded":"2024-04-25T11:38:26.761293+0000","last_fullsized":"2024-04-25T11:38:26.761293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673120+0000","last_change":"2024-04-25T11:38:25.672129+0000","last_active":"2024-04-25T11:38:26.673120+0000","last_peered":"2024-04-25T11:38:26.673120+0000","last_clean":"2024-04-25T11:38:26.673120+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:38:26.673120+0000","last_undegraded":"2024-04-25T11:38:26.673120+0000","last_fullsized":"2024-04-25T11:38:26.673120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761886+0000","last_change":"2024-04-25T11:38:25.682547+0000","last_active":"2024-04-25T11:38:26.761886+0000","last_peered":"2024-04-25T11:38:26.761886+0000","last_clean":"2024-04-25T11:38:26.761886+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:38:26.761886+0000","last_undegraded":"2024-04-25T11:38:26.761886+0000","last_fullsized":"2024-04-25T11:38:26.761886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763750+0000","last_change":"2024-04-25T11:38:25.683798+0000","last_active":"2024-04-25T11:38:26.763750+0000","last_peered":"2024-04-25T11:38:26.763750+0000","last_clean":"2024-04-25T11:38:26.763750+0000","last_became_active":"2024-04-25T11:38:25.682590+0000","last_became_peered":"2024-04-25T11:38:25.682590+0000","last_unstale":"2024-04-25T11:38:26.763750+0000","last_undegraded":"2024-04-25T11:38:26.763750+0000","last_fullsized":"2024-04-25T11:38:26.763750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673869+0000","last_change":"2024-04-25T11:38:25.669650+0000","last_active":"2024-04-25T11:38:26.673869+0000","last_peered":"2024-04-25T11:38:26.673869+0000","last_clean":"2024-04-25T11:38:26.673869+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:38:26.673869+0000","last_undegraded":"2024-04-25T11:38:26.673869+0000","last_fullsized":"2024-04-25T11:38:26.673869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762614+0000","last_change":"2024-04-25T11:38:25.678238+0000","last_active":"2024-04-25T11:38:26.762614+0000","last_peered":"2024-04-25T11:38:26.762614+0000","last_clean":"2024-04-25T11:38:26.762614+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:38:26.762614+0000","last_undegraded":"2024-04-25T11:38:26.762614+0000","last_fullsized":"2024-04-25T11:38:26.762614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674167+0000","last_change":"2024-04-25T11:38:25.666736+0000","last_active":"2024-04-25T11:38:26.674167+0000","last_peered":"2024-04-25T11:38:26.674167+0000","last_clean":"2024-04-25T11:38:26.674167+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:38:26.674167+0000","last_undegraded":"2024-04-25T11:38:26.674167+0000","last_fullsized":"2024-04-25T11:38:26.674167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761661+0000","last_change":"2024-04-25T11:38:25.680587+0000","last_active":"2024-04-25T11:38:26.761661+0000","last_peered":"2024-04-25T11:38:26.761661+0000","last_clean":"2024-04-25T11:38:26.761661+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:38:26.761661+0000","last_undegraded":"2024-04-25T11:38:26.761661+0000","last_fullsized":"2024-04-25T11:38:26.761661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762157+0000","last_change":"2024-04-25T11:38:25.680464+0000","last_active":"2024-04-25T11:38:26.762157+0000","last_peered":"2024-04-25T11:38:26.762157+0000","last_clean":"2024-04-25T11:38:26.762157+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:38:26.762157+0000","last_undegraded":"2024-04-25T11:38:26.762157+0000","last_fullsized":"2024-04-25T11:38:26.762157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761201+0000","last_change":"2024-04-25T11:38:25.672189+0000","last_active":"2024-04-25T11:38:26.761201+0000","last_peered":"2024-04-25T11:38:26.761201+0000","last_clean":"2024-04-25T11:38:26.761201+0000","last_became_active":"2024-04-25T11:38:25.671920+0000","last_became_peered":"2024-04-25T11:38:25.671920+0000","last_unstale":"2024-04-25T11:38:26.761201+0000","last_undegraded":"2024-04-25T11:38:26.761201+0000","last_fullsized":"2024-04-25T11:38:26.761201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762072+0000","last_change":"2024-04-25T11:38:25.674108+0000","last_active":"2024-04-25T11:38:26.762072+0000","last_peered":"2024-04-25T11:38:26.762072+0000","last_clean":"2024-04-25T11:38:26.762072+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:38:26.762072+0000","last_undegraded":"2024-04-25T11:38:26.762072+0000","last_fullsized":"2024-04-25T11:38:26.762072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761259+0000","last_change":"2024-04-25T11:38:25.672215+0000","last_active":"2024-04-25T11:38:26.761259+0000","last_peered":"2024-04-25T11:38:26.761259+0000","last_clean":"2024-04-25T11:38:26.761259+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:38:26.761259+0000","last_undegraded":"2024-04-25T11:38:26.761259+0000","last_fullsized":"2024-04-25T11:38:26.761259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674666+0000","last_change":"2024-04-25T11:38:25.679772+0000","last_active":"2024-04-25T11:38:26.674666+0000","last_peered":"2024-04-25T11:38:26.674666+0000","last_clean":"2024-04-25T11:38:26.674666+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:38:26.674666+0000","last_undegraded":"2024-04-25T11:38:26.674666+0000","last_fullsized":"2024-04-25T11:38:26.674666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673164+0000","last_change":"2024-04-25T11:38:25.665626+0000","last_active":"2024-04-25T11:38:26.673164+0000","last_peered":"2024-04-25T11:38:26.673164+0000","last_clean":"2024-04-25T11:38:26.673164+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:38:26.673164+0000","last_undegraded":"2024-04-25T11:38:26.673164+0000","last_fullsized":"2024-04-25T11:38:26.673164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761385+0000","last_change":"2024-04-25T11:38:25.672935+0000","last_active":"2024-04-25T11:38:26.761385+0000","last_peered":"2024-04-25T11:38:26.761385+0000","last_clean":"2024-04-25T11:38:26.761385+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:38:26.761385+0000","last_undegraded":"2024-04-25T11:38:26.761385+0000","last_fullsized":"2024-04-25T11:38:26.761385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761842+0000","last_change":"2024-04-25T11:38:25.679486+0000","last_active":"2024-04-25T11:38:26.761842+0000","last_peered":"2024-04-25T11:38:26.761842+0000","last_clean":"2024-04-25T11:38:26.761842+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:38:26.761842+0000","last_undegraded":"2024-04-25T11:38:26.761842+0000","last_fullsized":"2024-04-25T11:38:26.761842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762406+0000","last_change":"2024-04-25T11:38:25.679437+0000","last_active":"2024-04-25T11:38:26.762406+0000","last_peered":"2024-04-25T11:38:26.762406+0000","last_clean":"2024-04-25T11:38:26.762406+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:38:26.762406+0000","last_undegraded":"2024-04-25T11:38:26.762406+0000","last_fullsized":"2024-04-25T11:38:26.762406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673702+0000","last_change":"2024-04-25T11:38:25.670384+0000","last_active":"2024-04-25T11:38:26.673702+0000","last_peered":"2024-04-25T11:38:26.673702+0000","last_clean":"2024-04-25T11:38:26.673702+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:38:26.673702+0000","last_undegraded":"2024-04-25T11:38:26.673702+0000","last_fullsized":"2024-04-25T11:38:26.673702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674250+0000","last_change":"2024-04-25T11:38:25.680252+0000","last_active":"2024-04-25T11:38:26.674250+0000","last_peered":"2024-04-25T11:38:26.674250+0000","last_clean":"2024-04-25T11:38:26.674250+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:38:26.674250+0000","last_undegraded":"2024-04-25T11:38:26.674250+0000","last_fullsized":"2024-04-25T11:38:26.674250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762291+0000","last_change":"2024-04-25T11:38:25.669542+0000","last_active":"2024-04-25T11:38:26.762291+0000","last_peered":"2024-04-25T11:38:26.762291+0000","last_clean":"2024-04-25T11:38:26.762291+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:38:26.762291+0000","last_undegraded":"2024-04-25T11:38:26.762291+0000","last_fullsized":"2024-04-25T11:38:26.762291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:51.929330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.605314+0000","last_change":"2024-04-25T11:38:24.666197+0000","last_active":"2024-04-25T11:39:01.605314+0000","last_peered":"2024-04-25T11:39:01.605314+0000","last_clean":"2024-04-25T11:39:01.605314+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:39:01.605314+0000","last_undegraded":"2024-04-25T11:39:01.605314+0000","last_fullsized":"2024-04-25T11:39:01.605314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.198972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761974+0000","last_change":"2024-04-25T11:38:25.682931+0000","last_active":"2024-04-25T11:38:26.761974+0000","last_peered":"2024-04-25T11:38:26.761974+0000","last_clean":"2024-04-25T11:38:26.761974+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:38:26.761974+0000","last_undegraded":"2024-04-25T11:38:26.761974+0000","last_fullsized":"2024-04-25T11:38:26.761974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028972+0000","last_change":"2024-04-25T11:38:24.660888+0000","last_active":"2024-04-25T11:38:47.028972+0000","last_peered":"2024-04-25T11:38:47.028972+0000","last_clean":"2024-04-25T11:38:47.028972+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:38:47.028972+0000","last_undegraded":"2024-04-25T11:38:47.028972+0000","last_fullsized":"2024-04-25T11:38:47.028972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:41.729888+0000","last_change":"2024-04-25T11:38:24.657815+0000","last_active":"2024-04-25T11:38:41.729888+0000","last_peered":"2024-04-25T11:38:41.729888+0000","last_clean":"2024-04-25T11:38:41.729888+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:38:41.729888+0000","last_undegraded":"2024-04-25T11:38:41.729888+0000","last_fullsized":"2024-04-25T11:38:41.729888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761437+0000","last_change":"2024-04-25T11:38:25.680969+0000","last_active":"2024-04-25T11:38:26.761437+0000","last_peered":"2024-04-25T11:38:26.761437+0000","last_clean":"2024-04-25T11:38:26.761437+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:38:26.761437+0000","last_undegraded":"2024-04-25T11:38:26.761437+0000","last_fullsized":"2024-04-25T11:38:26.761437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028798+0000","last_change":"2024-04-25T11:38:24.668899+0000","last_active":"2024-04-25T11:38:47.028798+0000","last_peered":"2024-04-25T11:38:47.028798+0000","last_clean":"2024-04-25T11:38:47.028798+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:38:47.028798+0000","last_undegraded":"2024-04-25T11:38:47.028798+0000","last_fullsized":"2024-04-25T11:38:47.028798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672975+0000","last_change":"2024-04-25T11:38:25.666511+0000","last_active":"2024-04-25T11:38:26.672975+0000","last_peered":"2024-04-25T11:38:26.672975+0000","last_clean":"2024-04-25T11:38:26.672975+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:38:26.672975+0000","last_undegraded":"2024-04-25T11:38:26.672975+0000","last_fullsized":"2024-04-25T11:38:26.672975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.605159+0000","last_change":"2024-04-25T11:38:24.666735+0000","last_active":"2024-04-25T11:39:01.605159+0000","last_peered":"2024-04-25T11:39:01.605159+0000","last_clean":"2024-04-25T11:39:01.605159+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:39:01.605159+0000","last_undegraded":"2024-04-25T11:39:01.605159+0000","last_fullsized":"2024-04-25T11:39:01.605159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763586+0000","last_change":"2024-04-25T11:38:25.680065+0000","last_active":"2024-04-25T11:38:26.763586+0000","last_peered":"2024-04-25T11:38:26.763586+0000","last_clean":"2024-04-25T11:38:26.763586+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:38:26.763586+0000","last_undegraded":"2024-04-25T11:38:26.763586+0000","last_fullsized":"2024-04-25T11:38:26.763586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.048969+0000","last_change":"2024-04-25T11:38:24.671566+0000","last_active":"2024-04-25T11:38:53.048969+0000","last_peered":"2024-04-25T11:38:53.048969+0000","last_clean":"2024-04-25T11:38:53.048969+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:38:53.048969+0000","last_undegraded":"2024-04-25T11:38:53.048969+0000","last_fullsized":"2024-04-25T11:38:53.048969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761910+0000","last_change":"2024-04-25T11:38:25.678075+0000","last_active":"2024-04-25T11:38:26.761910+0000","last_peered":"2024-04-25T11:38:26.761910+0000","last_clean":"2024-04-25T11:38:26.761910+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:38:26.761910+0000","last_undegraded":"2024-04-25T11:38:26.761910+0000","last_fullsized":"2024-04-25T11:38:26.761910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:35.703616+0000","last_change":"2024-04-25T11:38:24.667282+0000","last_active":"2024-04-25T11:38:35.703616+0000","last_peered":"2024-04-25T11:38:35.703616+0000","last_clean":"2024-04-25T11:38:35.703616+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:38:35.703616+0000","last_undegraded":"2024-04-25T11:38:35.703616+0000","last_fullsized":"2024-04-25T11:38:35.703616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762833+0000","last_change":"2024-04-25T11:38:25.680417+0000","last_active":"2024-04-25T11:38:26.762833+0000","last_peered":"2024-04-25T11:38:26.762833+0000","last_clean":"2024-04-25T11:38:26.762833+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:38:26.762833+0000","last_undegraded":"2024-04-25T11:38:26.762833+0000","last_fullsized":"2024-04-25T11:38:26.762833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'14","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:05.634385+0000","last_change":"2024-04-25T11:38:24.662562+0000","last_active":"2024-04-25T11:39:05.634385+0000","last_peered":"2024-04-25T11:39:05.634385+0000","last_clean":"2024-04-25T11:39:05.634385+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:39:05.634385+0000","last_undegraded":"2024-04-25T11:39:05.634385+0000","last_fullsized":"2024-04-25T11:39:05.634385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762135+0000","last_change":"2024-04-25T11:38:25.677518+0000","last_active":"2024-04-25T11:38:26.762135+0000","last_peered":"2024-04-25T11:38:26.762135+0000","last_clean":"2024-04-25T11:38:26.762135+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:38:26.762135+0000","last_undegraded":"2024-04-25T11:38:26.762135+0000","last_fullsized":"2024-04-25T11:38:26.762135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.053184+0000","last_change":"2024-04-25T11:38:24.658960+0000","last_active":"2024-04-25T11:38:53.053184+0000","last_peered":"2024-04-25T11:38:53.053184+0000","last_clean":"2024-04-25T11:38:53.053184+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:38:53.053184+0000","last_undegraded":"2024-04-25T11:38:53.053184+0000","last_fullsized":"2024-04-25T11:38:53.053184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761438+0000","last_change":"2024-04-25T11:38:25.681355+0000","last_active":"2024-04-25T11:38:26.761438+0000","last_peered":"2024-04-25T11:38:26.761438+0000","last_clean":"2024-04-25T11:38:26.761438+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:38:26.761438+0000","last_undegraded":"2024-04-25T11:38:26.761438+0000","last_fullsized":"2024-04-25T11:38:26.761438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.029219+0000","last_change":"2024-04-25T11:38:24.666747+0000","last_active":"2024-04-25T11:38:47.029219+0000","last_peered":"2024-04-25T11:38:47.029219+0000","last_clean":"2024-04-25T11:38:47.029219+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:38:47.029219+0000","last_undegraded":"2024-04-25T11:38:47.029219+0000","last_fullsized":"2024-04-25T11:38:47.029219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762221+0000","last_change":"2024-04-25T11:38:25.684128+0000","last_active":"2024-04-25T11:38:26.762221+0000","last_peered":"2024-04-25T11:38:26.762221+0000","last_clean":"2024-04-25T11:38:26.762221+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:38:26.762221+0000","last_undegraded":"2024-04-25T11:38:26.762221+0000","last_fullsized":"2024-04-25T11:38:26.762221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.607906+0000","last_change":"2024-04-25T11:38:24.661103+0000","last_active":"2024-04-25T11:39:01.607906+0000","last_peered":"2024-04-25T11:39:01.607906+0000","last_clean":"2024-04-25T11:39:01.607906+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:39:01.607906+0000","last_undegraded":"2024-04-25T11:39:01.607906+0000","last_fullsized":"2024-04-25T11:39:01.607906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674380+0000","last_change":"2024-04-25T11:38:25.680490+0000","last_active":"2024-04-25T11:38:26.674380+0000","last_peered":"2024-04-25T11:38:26.674380+0000","last_clean":"2024-04-25T11:38:26.674380+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:38:26.674380+0000","last_undegraded":"2024-04-25T11:38:26.674380+0000","last_fullsized":"2024-04-25T11:38:26.674380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.024193+0000","last_change":"2024-04-25T11:38:24.665238+0000","last_active":"2024-04-25T11:38:47.024193+0000","last_peered":"2024-04-25T11:38:47.024193+0000","last_clean":"2024-04-25T11:38:47.024193+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:38:47.024193+0000","last_undegraded":"2024-04-25T11:38:47.024193+0000","last_fullsized":"2024-04-25T11:38:47.024193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761773+0000","last_change":"2024-04-25T11:38:25.682669+0000","last_active":"2024-04-25T11:38:26.761773+0000","last_peered":"2024-04-25T11:38:26.761773+0000","last_clean":"2024-04-25T11:38:26.761773+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:38:26.761773+0000","last_undegraded":"2024-04-25T11:38:26.761773+0000","last_fullsized":"2024-04-25T11:38:26.761773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679038+0000","last_change":"2024-04-25T11:38:24.658666+0000","last_active":"2024-04-25T11:38:30.679038+0000","last_peered":"2024-04-25T11:38:30.679038+0000","last_clean":"2024-04-25T11:38:30.679038+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:38:30.679038+0000","last_undegraded":"2024-04-25T11:38:30.679038+0000","last_fullsized":"2024-04-25T11:38:30.679038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:24.295273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762494+0000","last_change":"2024-04-25T11:38:25.680783+0000","last_active":"2024-04-25T11:38:26.762494+0000","last_peered":"2024-04-25T11:38:26.762494+0000","last_clean":"2024-04-25T11:38:26.762494+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:38:26.762494+0000","last_undegraded":"2024-04-25T11:38:26.762494+0000","last_fullsized":"2024-04-25T11:38:26.762494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.048152+0000","last_change":"2024-04-25T11:38:24.656937+0000","last_active":"2024-04-25T11:38:53.048152+0000","last_peered":"2024-04-25T11:38:53.048152+0000","last_clean":"2024-04-25T11:38:53.048152+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:38:53.048152+0000","last_undegraded":"2024-04-25T11:38:53.048152+0000","last_fullsized":"2024-04-25T11:38:53.048152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763039+0000","last_change":"2024-04-25T11:38:25.681307+0000","last_active":"2024-04-25T11:38:26.763039+0000","last_peered":"2024-04-25T11:38:26.763039+0000","last_clean":"2024-04-25T11:38:26.763039+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:38:26.763039+0000","last_undegraded":"2024-04-25T11:38:26.763039+0000","last_fullsized":"2024-04-25T11:38:26.763039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:17.678183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028781+0000","last_change":"2024-04-25T11:38:24.663247+0000","last_active":"2024-04-25T11:38:47.028781+0000","last_peered":"2024-04-25T11:38:47.028781+0000","last_clean":"2024-04-25T11:38:47.028781+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:38:47.028781+0000","last_undegraded":"2024-04-25T11:38:47.028781+0000","last_fullsized":"2024-04-25T11:38:47.028781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763176+0000","last_change":"2024-04-25T11:38:25.676409+0000","last_active":"2024-04-25T11:38:26.763176+0000","last_peered":"2024-04-25T11:38:26.763176+0000","last_clean":"2024-04-25T11:38:26.763176+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.763176+0000","last_undegraded":"2024-04-25T11:38:26.763176+0000","last_fullsized":"2024-04-25T11:38:26.763176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.602338+0000","last_change":"2024-04-25T11:38:24.666800+0000","last_active":"2024-04-25T11:39:01.602338+0000","last_peered":"2024-04-25T11:39:01.602338+0000","last_clean":"2024-04-25T11:39:01.602338+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:39:01.602338+0000","last_undegraded":"2024-04-25T11:39:01.602338+0000","last_fullsized":"2024-04-25T11:39:01.602338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762286+0000","last_change":"2024-04-25T11:38:25.678136+0000","last_active":"2024-04-25T11:38:26.762286+0000","last_peered":"2024-04-25T11:38:26.762286+0000","last_clean":"2024-04-25T11:38:26.762286+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:38:26.762286+0000","last_undegraded":"2024-04-25T11:38:26.762286+0000","last_fullsized":"2024-04-25T11:38:26.762286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761547+0000","last_change":"2024-04-25T11:38:20.865911+0000","last_active":"2024-04-25T11:38:26.761547+0000","last_peered":"2024-04-25T11:38:26.761547+0000","last_clean":"2024-04-25T11:38:26.761547+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:38:26.761547+0000","last_undegraded":"2024-04-25T11:38:26.761547+0000","last_fullsized":"2024-04-25T11:38:26.761547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_clean_scrub_stamp":"2024-04-25T11:38:19.606342+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:58.126842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.026102+0000","last_change":"2024-04-25T11:38:24.671283+0000","last_active":"2024-04-25T11:38:47.026102+0000","last_peered":"2024-04-25T11:38:47.026102+0000","last_clean":"2024-04-25T11:38:47.026102+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:38:47.026102+0000","last_undegraded":"2024-04-25T11:38:47.026102+0000","last_fullsized":"2024-04-25T11:38:47.026102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761567+0000","last_change":"2024-04-25T11:38:25.675002+0000","last_active":"2024-04-25T11:38:26.761567+0000","last_peered":"2024-04-25T11:38:26.761567+0000","last_clean":"2024-04-25T11:38:26.761567+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:38:26.761567+0000","last_undegraded":"2024-04-25T11:38:26.761567+0000","last_fullsized":"2024-04-25T11:38:26.761567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:41.726797+0000","last_change":"2024-04-25T11:38:24.671213+0000","last_active":"2024-04-25T11:38:41.726797+0000","last_peered":"2024-04-25T11:38:41.726797+0000","last_clean":"2024-04-25T11:38:41.726797+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:38:41.726797+0000","last_undegraded":"2024-04-25T11:38:41.726797+0000","last_fullsized":"2024-04-25T11:38:41.726797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761052+0000","last_change":"2024-04-25T11:38:25.673353+0000","last_active":"2024-04-25T11:38:26.761052+0000","last_peered":"2024-04-25T11:38:26.761052+0000","last_clean":"2024-04-25T11:38:26.761052+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:38:26.761052+0000","last_undegraded":"2024-04-25T11:38:26.761052+0000","last_fullsized":"2024-04-25T11:38:26.761052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:48.553601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.052041+0000","last_change":"2024-04-25T11:38:24.650599+0000","last_active":"2024-04-25T11:38:53.052041+0000","last_peered":"2024-04-25T11:38:53.052041+0000","last_clean":"2024-04-25T11:38:53.052041+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:38:53.052041+0000","last_undegraded":"2024-04-25T11:38:53.052041+0000","last_fullsized":"2024-04-25T11:38:53.052041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672982+0000","last_change":"2024-04-25T11:38:25.668956+0000","last_active":"2024-04-25T11:38:26.672982+0000","last_peered":"2024-04-25T11:38:26.672982+0000","last_clean":"2024-04-25T11:38:26.672982+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:38:26.672982+0000","last_undegraded":"2024-04-25T11:38:26.672982+0000","last_fullsized":"2024-04-25T11:38:26.672982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.053634+0000","last_change":"2024-04-25T11:38:24.665496+0000","last_active":"2024-04-25T11:38:53.053634+0000","last_peered":"2024-04-25T11:38:53.053634+0000","last_clean":"2024-04-25T11:38:53.053634+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:38:53.053634+0000","last_undegraded":"2024-04-25T11:38:53.053634+0000","last_fullsized":"2024-04-25T11:38:53.053634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762812+0000","last_change":"2024-04-25T11:38:25.681800+0000","last_active":"2024-04-25T11:38:26.762812+0000","last_peered":"2024-04-25T11:38:26.762812+0000","last_clean":"2024-04-25T11:38:26.762812+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:38:26.762812+0000","last_undegraded":"2024-04-25T11:38:26.762812+0000","last_fullsized":"2024-04-25T11:38:26.762812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:39.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.589274+0000","last_change":"2024-04-25T11:38:24.659792+0000","last_active":"2024-04-25T11:38:28.589274+0000","last_peered":"2024-04-25T11:38:28.589274+0000","last_clean":"2024-04-25T11:38:28.589274+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:38:28.589274+0000","last_undegraded":"2024-04-25T11:38:28.589274+0000","last_fullsized":"2024-04-25T11:38:28.589274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763580+0000","last_change":"2024-04-25T11:38:25.673411+0000","last_active":"2024-04-25T11:38:26.763580+0000","last_peered":"2024-04-25T11:38:26.763580+0000","last_clean":"2024-04-25T11:38:26.763580+0000","last_became_active":"2024-04-25T11:38:25.673116+0000","last_became_peered":"2024-04-25T11:38:25.673116+0000","last_unstale":"2024-04-25T11:38:26.763580+0000","last_undegraded":"2024-04-25T11:38:26.763580+0000","last_fullsized":"2024-04-25T11:38:26.763580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.024457+0000","last_change":"2024-04-25T11:38:24.671412+0000","last_active":"2024-04-25T11:38:47.024457+0000","last_peered":"2024-04-25T11:38:47.024457+0000","last_clean":"2024-04-25T11:38:47.024457+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:38:47.024457+0000","last_undegraded":"2024-04-25T11:38:47.024457+0000","last_fullsized":"2024-04-25T11:38:47.024457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:34.607841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673048+0000","last_change":"2024-04-25T11:38:25.668959+0000","last_active":"2024-04-25T11:38:26.673048+0000","last_peered":"2024-04-25T11:38:26.673048+0000","last_clean":"2024-04-25T11:38:26.673048+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:38:26.673048+0000","last_undegraded":"2024-04-25T11:38:26.673048+0000","last_fullsized":"2024-04-25T11:38:26.673048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028863+0000","last_change":"2024-04-25T11:38:24.664146+0000","last_active":"2024-04-25T11:38:47.028863+0000","last_peered":"2024-04-25T11:38:47.028863+0000","last_clean":"2024-04-25T11:38:47.028863+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:38:47.028863+0000","last_undegraded":"2024-04-25T11:38:47.028863+0000","last_fullsized":"2024-04-25T11:38:47.028863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761339+0000","last_change":"2024-04-25T11:38:25.680105+0000","last_active":"2024-04-25T11:38:26.761339+0000","last_peered":"2024-04-25T11:38:26.761339+0000","last_clean":"2024-04-25T11:38:26.761339+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:38:26.761339+0000","last_undegraded":"2024-04-25T11:38:26.761339+0000","last_fullsized":"2024-04-25T11:38:26.761339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.049396+0000","last_change":"2024-04-25T11:38:24.668369+0000","last_active":"2024-04-25T11:38:53.049396+0000","last_peered":"2024-04-25T11:38:53.049396+0000","last_clean":"2024-04-25T11:38:53.049396+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:38:53.049396+0000","last_undegraded":"2024-04-25T11:38:53.049396+0000","last_fullsized":"2024-04-25T11:38:53.049396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761723+0000","last_change":"2024-04-25T11:38:25.676446+0000","last_active":"2024-04-25T11:38:26.761723+0000","last_peered":"2024-04-25T11:38:26.761723+0000","last_clean":"2024-04-25T11:38:26.761723+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.761723+0000","last_undegraded":"2024-04-25T11:38:26.761723+0000","last_fullsized":"2024-04-25T11:38:26.761723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:35.700179+0000","last_change":"2024-04-25T11:38:24.671951+0000","last_active":"2024-04-25T11:38:35.700179+0000","last_peered":"2024-04-25T11:38:35.700179+0000","last_clean":"2024-04-25T11:38:35.700179+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:38:35.700179+0000","last_undegraded":"2024-04-25T11:38:35.700179+0000","last_fullsized":"2024-04-25T11:38:35.700179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761852+0000","last_change":"2024-04-25T11:38:25.669474+0000","last_active":"2024-04-25T11:38:26.761852+0000","last_peered":"2024-04-25T11:38:26.761852+0000","last_clean":"2024-04-25T11:38:26.761852+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:38:26.761852+0000","last_undegraded":"2024-04-25T11:38:26.761852+0000","last_fullsized":"2024-04-25T11:38:26.761852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:38.359208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.048585+0000","last_change":"2024-04-25T11:38:24.668689+0000","last_active":"2024-04-25T11:38:53.048585+0000","last_peered":"2024-04-25T11:38:53.048585+0000","last_clean":"2024-04-25T11:38:53.048585+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:38:53.048585+0000","last_undegraded":"2024-04-25T11:38:53.048585+0000","last_fullsized":"2024-04-25T11:38:53.048585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761421+0000","last_change":"2024-04-25T11:38:25.668088+0000","last_active":"2024-04-25T11:38:26.761421+0000","last_peered":"2024-04-25T11:38:26.761421+0000","last_clean":"2024-04-25T11:38:26.761421+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:38:26.761421+0000","last_undegraded":"2024-04-25T11:38:26.761421+0000","last_fullsized":"2024-04-25T11:38:26.761421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.029132+0000","last_change":"2024-04-25T11:38:24.662913+0000","last_active":"2024-04-25T11:38:47.029132+0000","last_peered":"2024-04-25T11:38:47.029132+0000","last_clean":"2024-04-25T11:38:47.029132+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:38:47.029132+0000","last_undegraded":"2024-04-25T11:38:47.029132+0000","last_fullsized":"2024-04-25T11:38:47.029132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761308+0000","last_change":"2024-04-25T11:38:25.680244+0000","last_active":"2024-04-25T11:38:26.761308+0000","last_peered":"2024-04-25T11:38:26.761308+0000","last_clean":"2024-04-25T11:38:26.761308+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:38:26.761308+0000","last_undegraded":"2024-04-25T11:38:26.761308+0000","last_fullsized":"2024-04-25T11:38:26.761308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:35.702098+0000","last_change":"2024-04-25T11:38:24.669196+0000","last_active":"2024-04-25T11:38:35.702098+0000","last_peered":"2024-04-25T11:38:35.702098+0000","last_clean":"2024-04-25T11:38:35.702098+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:38:35.702098+0000","last_undegraded":"2024-04-25T11:38:35.702098+0000","last_fullsized":"2024-04-25T11:38:35.702098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.791358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674198+0000","last_change":"2024-04-25T11:38:25.668400+0000","last_active":"2024-04-25T11:38:26.674198+0000","last_peered":"2024-04-25T11:38:26.674198+0000","last_clean":"2024-04-25T11:38:26.674198+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:38:26.674198+0000","last_undegraded":"2024-04-25T11:38:26.674198+0000","last_fullsized":"2024-04-25T11:38:26.674198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763520+0000","last_change":"2024-04-25T11:38:25.683663+0000","last_active":"2024-04-25T11:38:26.763520+0000","last_peered":"2024-04-25T11:38:26.763520+0000","last_clean":"2024-04-25T11:38:26.763520+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:38:26.763520+0000","last_undegraded":"2024-04-25T11:38:26.763520+0000","last_fullsized":"2024-04-25T11:38:26.763520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.049503+0000","last_change":"2024-04-25T11:38:24.670671+0000","last_active":"2024-04-25T11:38:53.049503+0000","last_peered":"2024-04-25T11:38:53.049503+0000","last_clean":"2024-04-25T11:38:53.049503+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:38:53.049503+0000","last_undegraded":"2024-04-25T11:38:53.049503+0000","last_fullsized":"2024-04-25T11:38:53.049503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763017+0000","last_change":"2024-04-25T11:38:25.681439+0000","last_active":"2024-04-25T11:38:26.763017+0000","last_peered":"2024-04-25T11:38:26.763017+0000","last_clean":"2024-04-25T11:38:26.763017+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:38:26.763017+0000","last_undegraded":"2024-04-25T11:38:26.763017+0000","last_fullsized":"2024-04-25T11:38:26.763017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.050209+0000","last_change":"2024-04-25T11:38:24.669485+0000","last_active":"2024-04-25T11:38:53.050209+0000","last_peered":"2024-04-25T11:38:53.050209+0000","last_clean":"2024-04-25T11:38:53.050209+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:38:53.050209+0000","last_undegraded":"2024-04-25T11:38:53.050209+0000","last_fullsized":"2024-04-25T11:38:53.050209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761766+0000","last_change":"2024-04-25T11:38:25.684415+0000","last_active":"2024-04-25T11:38:26.761766+0000","last_peered":"2024-04-25T11:38:26.761766+0000","last_clean":"2024-04-25T11:38:26.761766+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:38:26.761766+0000","last_undegraded":"2024-04-25T11:38:26.761766+0000","last_fullsized":"2024-04-25T11:38:26.761766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.678908+0000","last_change":"2024-04-25T11:38:24.658741+0000","last_active":"2024-04-25T11:38:30.678908+0000","last_peered":"2024-04-25T11:38:30.678908+0000","last_clean":"2024-04-25T11:38:30.678908+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:38:30.678908+0000","last_undegraded":"2024-04-25T11:38:30.678908+0000","last_fullsized":"2024-04-25T11:38:30.678908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761168+0000","last_change":"2024-04-25T11:38:25.667505+0000","last_active":"2024-04-25T11:38:26.761168+0000","last_peered":"2024-04-25T11:38:26.761168+0000","last_clean":"2024-04-25T11:38:26.761168+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:38:26.761168+0000","last_undegraded":"2024-04-25T11:38:26.761168+0000","last_fullsized":"2024-04-25T11:38:26.761168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.049060+0000","last_change":"2024-04-25T11:38:24.669215+0000","last_active":"2024-04-25T11:38:53.049060+0000","last_peered":"2024-04-25T11:38:53.049060+0000","last_clean":"2024-04-25T11:38:53.049060+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:38:53.049060+0000","last_undegraded":"2024-04-25T11:38:53.049060+0000","last_fullsized":"2024-04-25T11:38:53.049060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11177,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":218,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":22354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":663984,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":75315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":75448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":75349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":662176,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:39:12.678 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:39:12.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:39:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.828+0000 7fbf40d20640 1 Processor -- start 2024-04-25T11:39:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.829+0000 7fbf40d20640 1 -- start start 2024-04-25T11:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.829+0000 7fbf40d20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf3c155080 0x7fbf3c0cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.829+0000 7fbf40d20640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c0c7840 0x7fbf3c0d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.829+0000 7fbf40d20640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf3c0cb460 con 0x7fbf3c15ed40 2024-04-25T11:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.829+0000 7fbf40d20640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3c0cb740 con 0x7fbf3c155080 2024-04-25T11:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.829+0000 7fbf40d20640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c15af50 con 0x7fbf3c0c7840 2024-04-25T11:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.830+0000 7fbf3a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf3c155080 0x7fbf3c0cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.830+0000 7fbf3a59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf3c155080 0x7fbf3c0cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37582/0 (socket says 172.21.15.40:37582) 2024-04-25T11:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.830+0000 7fbf3a59c640 1 -- 172.21.15.40:0/443673667 learned_addr learned my addr 172.21.15.40:0/443673667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.830+0000 7fbf3ad9d640 1 --2- 172.21.15.40:0/443673667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c0c7840 0x7fbf3c0d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.830+0000 7fbf3a59c640 1 -- 172.21.15.40:0/443673667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c0c7840 msgr2=0x7fbf3c0d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.830+0000 7fbf3a59c640 1 --2- 172.21.15.40:0/443673667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c0c7840 0x7fbf3c0d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.830+0000 7fbf3a59c640 1 -- 172.21.15.40:0/443673667 >> v1:172.21.15.62:6789/0 conn(0x7fbf3c15ed40 legacy=0x7fbf3c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.830+0000 7fbf3a59c640 1 -- 172.21.15.40:0/443673667 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c15b230 con 0x7fbf3c155080 2024-04-25T11:39:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.830+0000 7fbf3ad9d640 1 --2- 172.21.15.40:0/443673667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c0c7840 0x7fbf3c0d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf3a59c640 1 --2- 172.21.15.40:0/443673667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c155080 0x7fbf3c0cc720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009870 tx=0x7fbf2c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e516d786ea8d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf3959a640 1 -- 172.21.15.40:0/443673667 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c037070 con 0x7fbf3c155080 2024-04-25T11:39:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf3959a640 1 -- 172.21.15.40:0/443673667 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf2c032440 con 0x7fbf3c155080 2024-04-25T11:39:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf3959a640 1 -- 172.21.15.40:0/443673667 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c032810 con 0x7fbf3c155080 2024-04-25T11:39:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf40d20640 1 -- 172.21.15.40:0/443673667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c155080 msgr2=0x7fbf3c0cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf40d20640 1 --2- 172.21.15.40:0/443673667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c155080 0x7fbf3c0cc720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009870 tx=0x7fbf2c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf40d20640 1 -- 172.21.15.40:0/443673667 shutdown_connections 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf40d20640 1 --2- 172.21.15.40:0/443673667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c0c7840 0x7fbf3c0d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf40d20640 1 --2- 172.21.15.40:0/443673667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c155080 0x7fbf3c0cc720 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.831+0000 7fbf40d20640 1 -- 172.21.15.40:0/443673667 >> 172.21.15.40:0/443673667 conn(0x7fbf3c06c680 msgr2=0x7fbf3c06e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.832+0000 7fbf40d20640 1 -- 172.21.15.40:0/443673667 shutdown_connections 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.832+0000 7fbf40d20640 1 -- 172.21.15.40:0/443673667 wait complete. 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.833+0000 7fbf40d20640 1 Processor -- start 2024-04-25T11:39:12.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.833+0000 7fbf40d20640 1 -- start start 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.833+0000 7fbf40d20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf3c0c7840 0x7fbf3c0c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf40d20640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c155080 0x7fbf3c0c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf40d20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c15ed40 0x7fbf3c0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf40d20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c0cc140 con 0x7fbf3c15ed40 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf40d20640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c15f210 con 0x7fbf3c0c7840 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf40d20640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c0cc2c0 con 0x7fbf3c155080 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c15ed40 0x7fbf3c0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c15ed40 0x7fbf3c0d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37584/0 (socket says 172.21.15.40:37584) 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3ad9d640 1 -- 172.21.15.40:0/2749031818 learned_addr learned my addr 172.21.15.40:0/2749031818 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3a59c640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf3c0c7840 0x7fbf3c0c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf39d9b640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c155080 0x7fbf3c0c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3ad9d640 1 -- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c155080 msgr2=0x7fbf3c0c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3ad9d640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c155080 0x7fbf3c0c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3ad9d640 1 -- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf3c0c7840 msgr2=0x7fbf3c0c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3ad9d640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf3c0c7840 0x7fbf3c0c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3ad9d640 1 -- 172.21.15.40:0/2749031818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c157ed0 con 0x7fbf3c15ed40 2024-04-25T11:39:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf39d9b640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c155080 0x7fbf3c0c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf3ad9d640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c15ed40 0x7fbf3c0d0100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf34007ef0 tx=0x7fbf3400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf2affd640 1 -- 172.21.15.40:0/2749031818 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3401a070 con 0x7fbf3c15ed40 2024-04-25T11:39:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf2affd640 1 -- 172.21.15.40:0/2749031818 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf34003b90 con 0x7fbf3c15ed40 2024-04-25T11:39:12.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c1581b0 con 0x7fbf3c15ed40 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf2affd640 1 -- 172.21.15.40:0/2749031818 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400de20 con 0x7fbf3c15ed40 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.834+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c0c4350 con 0x7fbf3c15ed40 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.835+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3c1586d0 con 0x7fbf3c15ed40 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.836+0000 7fbf2affd640 1 -- 172.21.15.40:0/2749031818 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf3401d070 con 0x7fbf3c15ed40 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.836+0000 7fbf2affd640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbf1c06c920 0x7fbf1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.836+0000 7fbf2affd640 1 -- 172.21.15.40:0/2749031818 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbf3408bd10 con 0x7fbf3c15ed40 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.836+0000 7fbf3a59c640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbf1c06c920 0x7fbf1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.837+0000 7fbf3a59c640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbf1c06c920 0x7fbf1c06ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c034000 tx=0x7fbf2c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:12.841+0000 7fbf2affd640 1 -- 172.21.15.40:0/2749031818 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf3c1586d0 con 0x7fbf3c15ed40 2024-04-25T11:39:13.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.023+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf3c0c4190 con 0x7fbf1c06c920 2024-04-25T11:39:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.030+0000 7fbf2affd640 1 -- 172.21.15.40:0/2749031818 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252064 (secure 0 0 0) 0x7fbf2c0326f0 con 0x7fbf1c06c920 2024-04-25T11:39:13.020 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:13.021 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:39:13.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbf1c06c920 msgr2=0x7fbf1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbf1c06c920 0x7fbf1c06ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c034000 tx=0x7fbf2c02e000 comp rx=0 tx=0).stop 2024-04-25T11:39:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c15ed40 msgr2=0x7fbf3c0d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c15ed40 0x7fbf3c0d0100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf34007ef0 tx=0x7fbf3400d4a0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 shutdown_connections 2024-04-25T11:39:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbf1c06c920 0x7fbf1c06ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c15ed40 0x7fbf3c0d0100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf3c155080 0x7fbf3c0c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 --2- 172.21.15.40:0/2749031818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf3c0c7840 0x7fbf3c0c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 >> 172.21.15.40:0/2749031818 conn(0x7fbf3c06c680 msgr2=0x7fbf3c1542b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:13.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 shutdown_connections 2024-04-25T11:39:13.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.033+0000 7fbf40d20640 1 -- 172.21.15.40:0/2749031818 wait complete. 2024-04-25T11:39:13.031 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-04-25T11:39:12.959965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601545,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":377,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218112,"kb_used_data":1984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":2031616,"data_stored":1770562,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3143,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002477"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762879+0000","last_change":"2024-04-25T11:38:25.677572+0000","last_active":"2024-04-25T11:38:26.762879+0000","last_peered":"2024-04-25T11:38:26.762879+0000","last_clean":"2024-04-25T11:38:26.762879+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:38:26.762879+0000","last_undegraded":"2024-04-25T11:38:26.762879+0000","last_fullsized":"2024-04-25T11:38:26.762879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673367+0000","last_change":"2024-04-25T11:38:25.677437+0000","last_active":"2024-04-25T11:38:26.673367+0000","last_peered":"2024-04-25T11:38:26.673367+0000","last_clean":"2024-04-25T11:38:26.673367+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:38:26.673367+0000","last_undegraded":"2024-04-25T11:38:26.673367+0000","last_fullsized":"2024-04-25T11:38:26.673367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761847+0000","last_change":"2024-04-25T11:38:25.676553+0000","last_active":"2024-04-25T11:38:26.761847+0000","last_peered":"2024-04-25T11:38:26.761847+0000","last_clean":"2024-04-25T11:38:26.761847+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:38:26.761847+0000","last_undegraded":"2024-04-25T11:38:26.761847+0000","last_fullsized":"2024-04-25T11:38:26.761847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762862+0000","last_change":"2024-04-25T11:38:25.676518+0000","last_active":"2024-04-25T11:38:26.762862+0000","last_peered":"2024-04-25T11:38:26.762862+0000","last_clean":"2024-04-25T11:38:26.762862+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:38:26.762862+0000","last_undegraded":"2024-04-25T11:38:26.762862+0000","last_fullsized":"2024-04-25T11:38:26.762862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762062+0000","last_change":"2024-04-25T11:38:25.678524+0000","last_active":"2024-04-25T11:38:26.762062+0000","last_peered":"2024-04-25T11:38:26.762062+0000","last_clean":"2024-04-25T11:38:26.762062+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:38:26.762062+0000","last_undegraded":"2024-04-25T11:38:26.762062+0000","last_fullsized":"2024-04-25T11:38:26.762062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761774+0000","last_change":"2024-04-25T11:38:25.683983+0000","last_active":"2024-04-25T11:38:26.761774+0000","last_peered":"2024-04-25T11:38:26.761774+0000","last_clean":"2024-04-25T11:38:26.761774+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:38:26.761774+0000","last_undegraded":"2024-04-25T11:38:26.761774+0000","last_fullsized":"2024-04-25T11:38:26.761774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761720+0000","last_change":"2024-04-25T11:38:25.682904+0000","last_active":"2024-04-25T11:38:26.761720+0000","last_peered":"2024-04-25T11:38:26.761720+0000","last_clean":"2024-04-25T11:38:26.761720+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:38:26.761720+0000","last_undegraded":"2024-04-25T11:38:26.761720+0000","last_fullsized":"2024-04-25T11:38:26.761720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761274+0000","last_change":"2024-04-25T11:38:25.670312+0000","last_active":"2024-04-25T11:38:26.761274+0000","last_peered":"2024-04-25T11:38:26.761274+0000","last_clean":"2024-04-25T11:38:26.761274+0000","last_became_active":"2024-04-25T11:38:25.669988+0000","last_became_peered":"2024-04-25T11:38:25.669988+0000","last_unstale":"2024-04-25T11:38:26.761274+0000","last_undegraded":"2024-04-25T11:38:26.761274+0000","last_fullsized":"2024-04-25T11:38:26.761274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762939+0000","last_change":"2024-04-25T11:38:25.678459+0000","last_active":"2024-04-25T11:38:26.762939+0000","last_peered":"2024-04-25T11:38:26.762939+0000","last_clean":"2024-04-25T11:38:26.762939+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:38:26.762939+0000","last_undegraded":"2024-04-25T11:38:26.762939+0000","last_fullsized":"2024-04-25T11:38:26.762939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674629+0000","last_change":"2024-04-25T11:38:25.675561+0000","last_active":"2024-04-25T11:38:26.674629+0000","last_peered":"2024-04-25T11:38:26.674629+0000","last_clean":"2024-04-25T11:38:26.674629+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:38:26.674629+0000","last_undegraded":"2024-04-25T11:38:26.674629+0000","last_fullsized":"2024-04-25T11:38:26.674629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674251+0000","last_change":"2024-04-25T11:38:25.671113+0000","last_active":"2024-04-25T11:38:26.674251+0000","last_peered":"2024-04-25T11:38:26.674251+0000","last_clean":"2024-04-25T11:38:26.674251+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:38:26.674251+0000","last_undegraded":"2024-04-25T11:38:26.674251+0000","last_fullsized":"2024-04-25T11:38:26.674251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762821+0000","last_change":"2024-04-25T11:38:25.665690+0000","last_active":"2024-04-25T11:38:26.762821+0000","last_peered":"2024-04-25T11:38:26.762821+0000","last_clean":"2024-04-25T11:38:26.762821+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:38:26.762821+0000","last_undegraded":"2024-04-25T11:38:26.762821+0000","last_fullsized":"2024-04-25T11:38:26.762821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:25.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761152+0000","last_change":"2024-04-25T11:38:25.666858+0000","last_active":"2024-04-25T11:38:26.761152+0000","last_peered":"2024-04-25T11:38:26.761152+0000","last_clean":"2024-04-25T11:38:26.761152+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:38:26.761152+0000","last_undegraded":"2024-04-25T11:38:26.761152+0000","last_fullsized":"2024-04-25T11:38:26.761152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761293+0000","last_change":"2024-04-25T11:38:25.677561+0000","last_active":"2024-04-25T11:38:26.761293+0000","last_peered":"2024-04-25T11:38:26.761293+0000","last_clean":"2024-04-25T11:38:26.761293+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:38:26.761293+0000","last_undegraded":"2024-04-25T11:38:26.761293+0000","last_fullsized":"2024-04-25T11:38:26.761293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673120+0000","last_change":"2024-04-25T11:38:25.672129+0000","last_active":"2024-04-25T11:38:26.673120+0000","last_peered":"2024-04-25T11:38:26.673120+0000","last_clean":"2024-04-25T11:38:26.673120+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:38:26.673120+0000","last_undegraded":"2024-04-25T11:38:26.673120+0000","last_fullsized":"2024-04-25T11:38:26.673120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761886+0000","last_change":"2024-04-25T11:38:25.682547+0000","last_active":"2024-04-25T11:38:26.761886+0000","last_peered":"2024-04-25T11:38:26.761886+0000","last_clean":"2024-04-25T11:38:26.761886+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:38:26.761886+0000","last_undegraded":"2024-04-25T11:38:26.761886+0000","last_fullsized":"2024-04-25T11:38:26.761886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763750+0000","last_change":"2024-04-25T11:38:25.683798+0000","last_active":"2024-04-25T11:38:26.763750+0000","last_peered":"2024-04-25T11:38:26.763750+0000","last_clean":"2024-04-25T11:38:26.763750+0000","last_became_active":"2024-04-25T11:38:25.682590+0000","last_became_peered":"2024-04-25T11:38:25.682590+0000","last_unstale":"2024-04-25T11:38:26.763750+0000","last_undegraded":"2024-04-25T11:38:26.763750+0000","last_fullsized":"2024-04-25T11:38:26.763750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673869+0000","last_change":"2024-04-25T11:38:25.669650+0000","last_active":"2024-04-25T11:38:26.673869+0000","last_peered":"2024-04-25T11:38:26.673869+0000","last_clean":"2024-04-25T11:38:26.673869+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:38:26.673869+0000","last_undegraded":"2024-04-25T11:38:26.673869+0000","last_fullsized":"2024-04-25T11:38:26.673869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762614+0000","last_change":"2024-04-25T11:38:25.678238+0000","last_active":"2024-04-25T11:38:26.762614+0000","last_peered":"2024-04-25T11:38:26.762614+0000","last_clean":"2024-04-25T11:38:26.762614+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:38:26.762614+0000","last_undegraded":"2024-04-25T11:38:26.762614+0000","last_fullsized":"2024-04-25T11:38:26.762614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674167+0000","last_change":"2024-04-25T11:38:25.666736+0000","last_active":"2024-04-25T11:38:26.674167+0000","last_peered":"2024-04-25T11:38:26.674167+0000","last_clean":"2024-04-25T11:38:26.674167+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:38:26.674167+0000","last_undegraded":"2024-04-25T11:38:26.674167+0000","last_fullsized":"2024-04-25T11:38:26.674167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761661+0000","last_change":"2024-04-25T11:38:25.680587+0000","last_active":"2024-04-25T11:38:26.761661+0000","last_peered":"2024-04-25T11:38:26.761661+0000","last_clean":"2024-04-25T11:38:26.761661+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:38:26.761661+0000","last_undegraded":"2024-04-25T11:38:26.761661+0000","last_fullsized":"2024-04-25T11:38:26.761661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762157+0000","last_change":"2024-04-25T11:38:25.680464+0000","last_active":"2024-04-25T11:38:26.762157+0000","last_peered":"2024-04-25T11:38:26.762157+0000","last_clean":"2024-04-25T11:38:26.762157+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:38:26.762157+0000","last_undegraded":"2024-04-25T11:38:26.762157+0000","last_fullsized":"2024-04-25T11:38:26.762157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761201+0000","last_change":"2024-04-25T11:38:25.672189+0000","last_active":"2024-04-25T11:38:26.761201+0000","last_peered":"2024-04-25T11:38:26.761201+0000","last_clean":"2024-04-25T11:38:26.761201+0000","last_became_active":"2024-04-25T11:38:25.671920+0000","last_became_peered":"2024-04-25T11:38:25.671920+0000","last_unstale":"2024-04-25T11:38:26.761201+0000","last_undegraded":"2024-04-25T11:38:26.761201+0000","last_fullsized":"2024-04-25T11:38:26.761201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762072+0000","last_change":"2024-04-25T11:38:25.674108+0000","last_active":"2024-04-25T11:38:26.762072+0000","last_peered":"2024-04-25T11:38:26.762072+0000","last_clean":"2024-04-25T11:38:26.762072+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:38:26.762072+0000","last_undegraded":"2024-04-25T11:38:26.762072+0000","last_fullsized":"2024-04-25T11:38:26.762072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761259+0000","last_change":"2024-04-25T11:38:25.672215+0000","last_active":"2024-04-25T11:38:26.761259+0000","last_peered":"2024-04-25T11:38:26.761259+0000","last_clean":"2024-04-25T11:38:26.761259+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:38:26.761259+0000","last_undegraded":"2024-04-25T11:38:26.761259+0000","last_fullsized":"2024-04-25T11:38:26.761259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674666+0000","last_change":"2024-04-25T11:38:25.679772+0000","last_active":"2024-04-25T11:38:26.674666+0000","last_peered":"2024-04-25T11:38:26.674666+0000","last_clean":"2024-04-25T11:38:26.674666+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:38:26.674666+0000","last_undegraded":"2024-04-25T11:38:26.674666+0000","last_fullsized":"2024-04-25T11:38:26.674666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673164+0000","last_change":"2024-04-25T11:38:25.665626+0000","last_active":"2024-04-25T11:38:26.673164+0000","last_peered":"2024-04-25T11:38:26.673164+0000","last_clean":"2024-04-25T11:38:26.673164+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:38:26.673164+0000","last_undegraded":"2024-04-25T11:38:26.673164+0000","last_fullsized":"2024-04-25T11:38:26.673164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761385+0000","last_change":"2024-04-25T11:38:25.672935+0000","last_active":"2024-04-25T11:38:26.761385+0000","last_peered":"2024-04-25T11:38:26.761385+0000","last_clean":"2024-04-25T11:38:26.761385+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:38:26.761385+0000","last_undegraded":"2024-04-25T11:38:26.761385+0000","last_fullsized":"2024-04-25T11:38:26.761385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761842+0000","last_change":"2024-04-25T11:38:25.679486+0000","last_active":"2024-04-25T11:38:26.761842+0000","last_peered":"2024-04-25T11:38:26.761842+0000","last_clean":"2024-04-25T11:38:26.761842+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:38:26.761842+0000","last_undegraded":"2024-04-25T11:38:26.761842+0000","last_fullsized":"2024-04-25T11:38:26.761842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762406+0000","last_change":"2024-04-25T11:38:25.679437+0000","last_active":"2024-04-25T11:38:26.762406+0000","last_peered":"2024-04-25T11:38:26.762406+0000","last_clean":"2024-04-25T11:38:26.762406+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:38:26.762406+0000","last_undegraded":"2024-04-25T11:38:26.762406+0000","last_fullsized":"2024-04-25T11:38:26.762406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673702+0000","last_change":"2024-04-25T11:38:25.670384+0000","last_active":"2024-04-25T11:38:26.673702+0000","last_peered":"2024-04-25T11:38:26.673702+0000","last_clean":"2024-04-25T11:38:26.673702+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:38:26.673702+0000","last_undegraded":"2024-04-25T11:38:26.673702+0000","last_fullsized":"2024-04-25T11:38:26.673702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674250+0000","last_change":"2024-04-25T11:38:25.680252+0000","last_active":"2024-04-25T11:38:26.674250+0000","last_peered":"2024-04-25T11:38:26.674250+0000","last_clean":"2024-04-25T11:38:26.674250+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:38:26.674250+0000","last_undegraded":"2024-04-25T11:38:26.674250+0000","last_fullsized":"2024-04-25T11:38:26.674250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762291+0000","last_change":"2024-04-25T11:38:25.669542+0000","last_active":"2024-04-25T11:38:26.762291+0000","last_peered":"2024-04-25T11:38:26.762291+0000","last_clean":"2024-04-25T11:38:26.762291+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:38:26.762291+0000","last_undegraded":"2024-04-25T11:38:26.762291+0000","last_fullsized":"2024-04-25T11:38:26.762291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:51.929330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.605314+0000","last_change":"2024-04-25T11:38:24.666197+0000","last_active":"2024-04-25T11:39:01.605314+0000","last_peered":"2024-04-25T11:39:01.605314+0000","last_clean":"2024-04-25T11:39:01.605314+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:39:01.605314+0000","last_undegraded":"2024-04-25T11:39:01.605314+0000","last_fullsized":"2024-04-25T11:39:01.605314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.198972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761974+0000","last_change":"2024-04-25T11:38:25.682931+0000","last_active":"2024-04-25T11:38:26.761974+0000","last_peered":"2024-04-25T11:38:26.761974+0000","last_clean":"2024-04-25T11:38:26.761974+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:38:26.761974+0000","last_undegraded":"2024-04-25T11:38:26.761974+0000","last_fullsized":"2024-04-25T11:38:26.761974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028972+0000","last_change":"2024-04-25T11:38:24.660888+0000","last_active":"2024-04-25T11:38:47.028972+0000","last_peered":"2024-04-25T11:38:47.028972+0000","last_clean":"2024-04-25T11:38:47.028972+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:38:47.028972+0000","last_undegraded":"2024-04-25T11:38:47.028972+0000","last_fullsized":"2024-04-25T11:38:47.028972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:41.729888+0000","last_change":"2024-04-25T11:38:24.657815+0000","last_active":"2024-04-25T11:38:41.729888+0000","last_peered":"2024-04-25T11:38:41.729888+0000","last_clean":"2024-04-25T11:38:41.729888+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:38:41.729888+0000","last_undegraded":"2024-04-25T11:38:41.729888+0000","last_fullsized":"2024-04-25T11:38:41.729888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761437+0000","last_change":"2024-04-25T11:38:25.680969+0000","last_active":"2024-04-25T11:38:26.761437+0000","last_peered":"2024-04-25T11:38:26.761437+0000","last_clean":"2024-04-25T11:38:26.761437+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:38:26.761437+0000","last_undegraded":"2024-04-25T11:38:26.761437+0000","last_fullsized":"2024-04-25T11:38:26.761437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028798+0000","last_change":"2024-04-25T11:38:24.668899+0000","last_active":"2024-04-25T11:38:47.028798+0000","last_peered":"2024-04-25T11:38:47.028798+0000","last_clean":"2024-04-25T11:38:47.028798+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:38:47.028798+0000","last_undegraded":"2024-04-25T11:38:47.028798+0000","last_fullsized":"2024-04-25T11:38:47.028798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672975+0000","last_change":"2024-04-25T11:38:25.666511+0000","last_active":"2024-04-25T11:38:26.672975+0000","last_peered":"2024-04-25T11:38:26.672975+0000","last_clean":"2024-04-25T11:38:26.672975+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:38:26.672975+0000","last_undegraded":"2024-04-25T11:38:26.672975+0000","last_fullsized":"2024-04-25T11:38:26.672975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.605159+0000","last_change":"2024-04-25T11:38:24.666735+0000","last_active":"2024-04-25T11:39:01.605159+0000","last_peered":"2024-04-25T11:39:01.605159+0000","last_clean":"2024-04-25T11:39:01.605159+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:39:01.605159+0000","last_undegraded":"2024-04-25T11:39:01.605159+0000","last_fullsized":"2024-04-25T11:39:01.605159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763586+0000","last_change":"2024-04-25T11:38:25.680065+0000","last_active":"2024-04-25T11:38:26.763586+0000","last_peered":"2024-04-25T11:38:26.763586+0000","last_clean":"2024-04-25T11:38:26.763586+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:38:26.763586+0000","last_undegraded":"2024-04-25T11:38:26.763586+0000","last_fullsized":"2024-04-25T11:38:26.763586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.048969+0000","last_change":"2024-04-25T11:38:24.671566+0000","last_active":"2024-04-25T11:38:53.048969+0000","last_peered":"2024-04-25T11:38:53.048969+0000","last_clean":"2024-04-25T11:38:53.048969+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:38:53.048969+0000","last_undegraded":"2024-04-25T11:38:53.048969+0000","last_fullsized":"2024-04-25T11:38:53.048969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761910+0000","last_change":"2024-04-25T11:38:25.678075+0000","last_active":"2024-04-25T11:38:26.761910+0000","last_peered":"2024-04-25T11:38:26.761910+0000","last_clean":"2024-04-25T11:38:26.761910+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:38:26.761910+0000","last_undegraded":"2024-04-25T11:38:26.761910+0000","last_fullsized":"2024-04-25T11:38:26.761910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:35.703616+0000","last_change":"2024-04-25T11:38:24.667282+0000","last_active":"2024-04-25T11:38:35.703616+0000","last_peered":"2024-04-25T11:38:35.703616+0000","last_clean":"2024-04-25T11:38:35.703616+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:38:35.703616+0000","last_undegraded":"2024-04-25T11:38:35.703616+0000","last_fullsized":"2024-04-25T11:38:35.703616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762833+0000","last_change":"2024-04-25T11:38:25.680417+0000","last_active":"2024-04-25T11:38:26.762833+0000","last_peered":"2024-04-25T11:38:26.762833+0000","last_clean":"2024-04-25T11:38:26.762833+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:38:26.762833+0000","last_undegraded":"2024-04-25T11:38:26.762833+0000","last_fullsized":"2024-04-25T11:38:26.762833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'14","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:05.634385+0000","last_change":"2024-04-25T11:38:24.662562+0000","last_active":"2024-04-25T11:39:05.634385+0000","last_peered":"2024-04-25T11:39:05.634385+0000","last_clean":"2024-04-25T11:39:05.634385+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:39:05.634385+0000","last_undegraded":"2024-04-25T11:39:05.634385+0000","last_fullsized":"2024-04-25T11:39:05.634385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762135+0000","last_change":"2024-04-25T11:38:25.677518+0000","last_active":"2024-04-25T11:38:26.762135+0000","last_peered":"2024-04-25T11:38:26.762135+0000","last_clean":"2024-04-25T11:38:26.762135+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:38:26.762135+0000","last_undegraded":"2024-04-25T11:38:26.762135+0000","last_fullsized":"2024-04-25T11:38:26.762135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.053184+0000","last_change":"2024-04-25T11:38:24.658960+0000","last_active":"2024-04-25T11:38:53.053184+0000","last_peered":"2024-04-25T11:38:53.053184+0000","last_clean":"2024-04-25T11:38:53.053184+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:38:53.053184+0000","last_undegraded":"2024-04-25T11:38:53.053184+0000","last_fullsized":"2024-04-25T11:38:53.053184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761438+0000","last_change":"2024-04-25T11:38:25.681355+0000","last_active":"2024-04-25T11:38:26.761438+0000","last_peered":"2024-04-25T11:38:26.761438+0000","last_clean":"2024-04-25T11:38:26.761438+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:38:26.761438+0000","last_undegraded":"2024-04-25T11:38:26.761438+0000","last_fullsized":"2024-04-25T11:38:26.761438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.029219+0000","last_change":"2024-04-25T11:38:24.666747+0000","last_active":"2024-04-25T11:38:47.029219+0000","last_peered":"2024-04-25T11:38:47.029219+0000","last_clean":"2024-04-25T11:38:47.029219+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:38:47.029219+0000","last_undegraded":"2024-04-25T11:38:47.029219+0000","last_fullsized":"2024-04-25T11:38:47.029219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762221+0000","last_change":"2024-04-25T11:38:25.684128+0000","last_active":"2024-04-25T11:38:26.762221+0000","last_peered":"2024-04-25T11:38:26.762221+0000","last_clean":"2024-04-25T11:38:26.762221+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:38:26.762221+0000","last_undegraded":"2024-04-25T11:38:26.762221+0000","last_fullsized":"2024-04-25T11:38:26.762221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.607906+0000","last_change":"2024-04-25T11:38:24.661103+0000","last_active":"2024-04-25T11:39:01.607906+0000","last_peered":"2024-04-25T11:39:01.607906+0000","last_clean":"2024-04-25T11:39:01.607906+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:39:01.607906+0000","last_undegraded":"2024-04-25T11:39:01.607906+0000","last_fullsized":"2024-04-25T11:39:01.607906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674380+0000","last_change":"2024-04-25T11:38:25.680490+0000","last_active":"2024-04-25T11:38:26.674380+0000","last_peered":"2024-04-25T11:38:26.674380+0000","last_clean":"2024-04-25T11:38:26.674380+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:38:26.674380+0000","last_undegraded":"2024-04-25T11:38:26.674380+0000","last_fullsized":"2024-04-25T11:38:26.674380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.024193+0000","last_change":"2024-04-25T11:38:24.665238+0000","last_active":"2024-04-25T11:38:47.024193+0000","last_peered":"2024-04-25T11:38:47.024193+0000","last_clean":"2024-04-25T11:38:47.024193+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:38:47.024193+0000","last_undegraded":"2024-04-25T11:38:47.024193+0000","last_fullsized":"2024-04-25T11:38:47.024193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761773+0000","last_change":"2024-04-25T11:38:25.682669+0000","last_active":"2024-04-25T11:38:26.761773+0000","last_peered":"2024-04-25T11:38:26.761773+0000","last_clean":"2024-04-25T11:38:26.761773+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:38:26.761773+0000","last_undegraded":"2024-04-25T11:38:26.761773+0000","last_fullsized":"2024-04-25T11:38:26.761773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.679038+0000","last_change":"2024-04-25T11:38:24.658666+0000","last_active":"2024-04-25T11:38:30.679038+0000","last_peered":"2024-04-25T11:38:30.679038+0000","last_clean":"2024-04-25T11:38:30.679038+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:38:30.679038+0000","last_undegraded":"2024-04-25T11:38:30.679038+0000","last_fullsized":"2024-04-25T11:38:30.679038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:24.295273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762494+0000","last_change":"2024-04-25T11:38:25.680783+0000","last_active":"2024-04-25T11:38:26.762494+0000","last_peered":"2024-04-25T11:38:26.762494+0000","last_clean":"2024-04-25T11:38:26.762494+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:38:26.762494+0000","last_undegraded":"2024-04-25T11:38:26.762494+0000","last_fullsized":"2024-04-25T11:38:26.762494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.048152+0000","last_change":"2024-04-25T11:38:24.656937+0000","last_active":"2024-04-25T11:38:53.048152+0000","last_peered":"2024-04-25T11:38:53.048152+0000","last_clean":"2024-04-25T11:38:53.048152+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:38:53.048152+0000","last_undegraded":"2024-04-25T11:38:53.048152+0000","last_fullsized":"2024-04-25T11:38:53.048152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763039+0000","last_change":"2024-04-25T11:38:25.681307+0000","last_active":"2024-04-25T11:38:26.763039+0000","last_peered":"2024-04-25T11:38:26.763039+0000","last_clean":"2024-04-25T11:38:26.763039+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:38:26.763039+0000","last_undegraded":"2024-04-25T11:38:26.763039+0000","last_fullsized":"2024-04-25T11:38:26.763039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:17.678183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028781+0000","last_change":"2024-04-25T11:38:24.663247+0000","last_active":"2024-04-25T11:38:47.028781+0000","last_peered":"2024-04-25T11:38:47.028781+0000","last_clean":"2024-04-25T11:38:47.028781+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:38:47.028781+0000","last_undegraded":"2024-04-25T11:38:47.028781+0000","last_fullsized":"2024-04-25T11:38:47.028781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763176+0000","last_change":"2024-04-25T11:38:25.676409+0000","last_active":"2024-04-25T11:38:26.763176+0000","last_peered":"2024-04-25T11:38:26.763176+0000","last_clean":"2024-04-25T11:38:26.763176+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.763176+0000","last_undegraded":"2024-04-25T11:38:26.763176+0000","last_fullsized":"2024-04-25T11:38:26.763176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:39:01.602338+0000","last_change":"2024-04-25T11:38:24.666800+0000","last_active":"2024-04-25T11:39:01.602338+0000","last_peered":"2024-04-25T11:39:01.602338+0000","last_clean":"2024-04-25T11:39:01.602338+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:39:01.602338+0000","last_undegraded":"2024-04-25T11:39:01.602338+0000","last_fullsized":"2024-04-25T11:39:01.602338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762286+0000","last_change":"2024-04-25T11:38:25.678136+0000","last_active":"2024-04-25T11:38:26.762286+0000","last_peered":"2024-04-25T11:38:26.762286+0000","last_clean":"2024-04-25T11:38:26.762286+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:38:26.762286+0000","last_undegraded":"2024-04-25T11:38:26.762286+0000","last_fullsized":"2024-04-25T11:38:26.762286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761547+0000","last_change":"2024-04-25T11:38:20.865911+0000","last_active":"2024-04-25T11:38:26.761547+0000","last_peered":"2024-04-25T11:38:26.761547+0000","last_clean":"2024-04-25T11:38:26.761547+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:38:26.761547+0000","last_undegraded":"2024-04-25T11:38:26.761547+0000","last_fullsized":"2024-04-25T11:38:26.761547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_clean_scrub_stamp":"2024-04-25T11:38:19.606342+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:58.126842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.026102+0000","last_change":"2024-04-25T11:38:24.671283+0000","last_active":"2024-04-25T11:38:47.026102+0000","last_peered":"2024-04-25T11:38:47.026102+0000","last_clean":"2024-04-25T11:38:47.026102+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:38:47.026102+0000","last_undegraded":"2024-04-25T11:38:47.026102+0000","last_fullsized":"2024-04-25T11:38:47.026102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761567+0000","last_change":"2024-04-25T11:38:25.675002+0000","last_active":"2024-04-25T11:38:26.761567+0000","last_peered":"2024-04-25T11:38:26.761567+0000","last_clean":"2024-04-25T11:38:26.761567+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:38:26.761567+0000","last_undegraded":"2024-04-25T11:38:26.761567+0000","last_fullsized":"2024-04-25T11:38:26.761567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:41.726797+0000","last_change":"2024-04-25T11:38:24.671213+0000","last_active":"2024-04-25T11:38:41.726797+0000","last_peered":"2024-04-25T11:38:41.726797+0000","last_clean":"2024-04-25T11:38:41.726797+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:38:41.726797+0000","last_undegraded":"2024-04-25T11:38:41.726797+0000","last_fullsized":"2024-04-25T11:38:41.726797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761052+0000","last_change":"2024-04-25T11:38:25.673353+0000","last_active":"2024-04-25T11:38:26.761052+0000","last_peered":"2024-04-25T11:38:26.761052+0000","last_clean":"2024-04-25T11:38:26.761052+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:38:26.761052+0000","last_undegraded":"2024-04-25T11:38:26.761052+0000","last_fullsized":"2024-04-25T11:38:26.761052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:48.553601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.052041+0000","last_change":"2024-04-25T11:38:24.650599+0000","last_active":"2024-04-25T11:38:53.052041+0000","last_peered":"2024-04-25T11:38:53.052041+0000","last_clean":"2024-04-25T11:38:53.052041+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:38:53.052041+0000","last_undegraded":"2024-04-25T11:38:53.052041+0000","last_fullsized":"2024-04-25T11:38:53.052041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.672982+0000","last_change":"2024-04-25T11:38:25.668956+0000","last_active":"2024-04-25T11:38:26.672982+0000","last_peered":"2024-04-25T11:38:26.672982+0000","last_clean":"2024-04-25T11:38:26.672982+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:38:26.672982+0000","last_undegraded":"2024-04-25T11:38:26.672982+0000","last_fullsized":"2024-04-25T11:38:26.672982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.053634+0000","last_change":"2024-04-25T11:38:24.665496+0000","last_active":"2024-04-25T11:38:53.053634+0000","last_peered":"2024-04-25T11:38:53.053634+0000","last_clean":"2024-04-25T11:38:53.053634+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:38:53.053634+0000","last_undegraded":"2024-04-25T11:38:53.053634+0000","last_fullsized":"2024-04-25T11:38:53.053634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.762812+0000","last_change":"2024-04-25T11:38:25.681800+0000","last_active":"2024-04-25T11:38:26.762812+0000","last_peered":"2024-04-25T11:38:26.762812+0000","last_clean":"2024-04-25T11:38:26.762812+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:38:26.762812+0000","last_undegraded":"2024-04-25T11:38:26.762812+0000","last_fullsized":"2024-04-25T11:38:26.762812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:39.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:28.589274+0000","last_change":"2024-04-25T11:38:24.659792+0000","last_active":"2024-04-25T11:38:28.589274+0000","last_peered":"2024-04-25T11:38:28.589274+0000","last_clean":"2024-04-25T11:38:28.589274+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:38:28.589274+0000","last_undegraded":"2024-04-25T11:38:28.589274+0000","last_fullsized":"2024-04-25T11:38:28.589274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763580+0000","last_change":"2024-04-25T11:38:25.673411+0000","last_active":"2024-04-25T11:38:26.763580+0000","last_peered":"2024-04-25T11:38:26.763580+0000","last_clean":"2024-04-25T11:38:26.763580+0000","last_became_active":"2024-04-25T11:38:25.673116+0000","last_became_peered":"2024-04-25T11:38:25.673116+0000","last_unstale":"2024-04-25T11:38:26.763580+0000","last_undegraded":"2024-04-25T11:38:26.763580+0000","last_fullsized":"2024-04-25T11:38:26.763580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.024457+0000","last_change":"2024-04-25T11:38:24.671412+0000","last_active":"2024-04-25T11:38:47.024457+0000","last_peered":"2024-04-25T11:38:47.024457+0000","last_clean":"2024-04-25T11:38:47.024457+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:38:47.024457+0000","last_undegraded":"2024-04-25T11:38:47.024457+0000","last_fullsized":"2024-04-25T11:38:47.024457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:34.607841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.673048+0000","last_change":"2024-04-25T11:38:25.668959+0000","last_active":"2024-04-25T11:38:26.673048+0000","last_peered":"2024-04-25T11:38:26.673048+0000","last_clean":"2024-04-25T11:38:26.673048+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:38:26.673048+0000","last_undegraded":"2024-04-25T11:38:26.673048+0000","last_fullsized":"2024-04-25T11:38:26.673048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.028863+0000","last_change":"2024-04-25T11:38:24.664146+0000","last_active":"2024-04-25T11:38:47.028863+0000","last_peered":"2024-04-25T11:38:47.028863+0000","last_clean":"2024-04-25T11:38:47.028863+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:38:47.028863+0000","last_undegraded":"2024-04-25T11:38:47.028863+0000","last_fullsized":"2024-04-25T11:38:47.028863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761339+0000","last_change":"2024-04-25T11:38:25.680105+0000","last_active":"2024-04-25T11:38:26.761339+0000","last_peered":"2024-04-25T11:38:26.761339+0000","last_clean":"2024-04-25T11:38:26.761339+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:38:26.761339+0000","last_undegraded":"2024-04-25T11:38:26.761339+0000","last_fullsized":"2024-04-25T11:38:26.761339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.049396+0000","last_change":"2024-04-25T11:38:24.668369+0000","last_active":"2024-04-25T11:38:53.049396+0000","last_peered":"2024-04-25T11:38:53.049396+0000","last_clean":"2024-04-25T11:38:53.049396+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:38:53.049396+0000","last_undegraded":"2024-04-25T11:38:53.049396+0000","last_fullsized":"2024-04-25T11:38:53.049396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761723+0000","last_change":"2024-04-25T11:38:25.676446+0000","last_active":"2024-04-25T11:38:26.761723+0000","last_peered":"2024-04-25T11:38:26.761723+0000","last_clean":"2024-04-25T11:38:26.761723+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:38:26.761723+0000","last_undegraded":"2024-04-25T11:38:26.761723+0000","last_fullsized":"2024-04-25T11:38:26.761723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:35.700179+0000","last_change":"2024-04-25T11:38:24.671951+0000","last_active":"2024-04-25T11:38:35.700179+0000","last_peered":"2024-04-25T11:38:35.700179+0000","last_clean":"2024-04-25T11:38:35.700179+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:38:35.700179+0000","last_undegraded":"2024-04-25T11:38:35.700179+0000","last_fullsized":"2024-04-25T11:38:35.700179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761852+0000","last_change":"2024-04-25T11:38:25.669474+0000","last_active":"2024-04-25T11:38:26.761852+0000","last_peered":"2024-04-25T11:38:26.761852+0000","last_clean":"2024-04-25T11:38:26.761852+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:38:26.761852+0000","last_undegraded":"2024-04-25T11:38:26.761852+0000","last_fullsized":"2024-04-25T11:38:26.761852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:38.359208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.048585+0000","last_change":"2024-04-25T11:38:24.668689+0000","last_active":"2024-04-25T11:38:53.048585+0000","last_peered":"2024-04-25T11:38:53.048585+0000","last_clean":"2024-04-25T11:38:53.048585+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:38:53.048585+0000","last_undegraded":"2024-04-25T11:38:53.048585+0000","last_fullsized":"2024-04-25T11:38:53.048585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761421+0000","last_change":"2024-04-25T11:38:25.668088+0000","last_active":"2024-04-25T11:38:26.761421+0000","last_peered":"2024-04-25T11:38:26.761421+0000","last_clean":"2024-04-25T11:38:26.761421+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:38:26.761421+0000","last_undegraded":"2024-04-25T11:38:26.761421+0000","last_fullsized":"2024-04-25T11:38:26.761421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:47.029132+0000","last_change":"2024-04-25T11:38:24.662913+0000","last_active":"2024-04-25T11:38:47.029132+0000","last_peered":"2024-04-25T11:38:47.029132+0000","last_clean":"2024-04-25T11:38:47.029132+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:38:47.029132+0000","last_undegraded":"2024-04-25T11:38:47.029132+0000","last_fullsized":"2024-04-25T11:38:47.029132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761308+0000","last_change":"2024-04-25T11:38:25.680244+0000","last_active":"2024-04-25T11:38:26.761308+0000","last_peered":"2024-04-25T11:38:26.761308+0000","last_clean":"2024-04-25T11:38:26.761308+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:38:26.761308+0000","last_undegraded":"2024-04-25T11:38:26.761308+0000","last_fullsized":"2024-04-25T11:38:26.761308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:35.702098+0000","last_change":"2024-04-25T11:38:24.669196+0000","last_active":"2024-04-25T11:38:35.702098+0000","last_peered":"2024-04-25T11:38:35.702098+0000","last_clean":"2024-04-25T11:38:35.702098+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:38:35.702098+0000","last_undegraded":"2024-04-25T11:38:35.702098+0000","last_fullsized":"2024-04-25T11:38:35.702098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.791358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.674198+0000","last_change":"2024-04-25T11:38:25.668400+0000","last_active":"2024-04-25T11:38:26.674198+0000","last_peered":"2024-04-25T11:38:26.674198+0000","last_clean":"2024-04-25T11:38:26.674198+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:38:26.674198+0000","last_undegraded":"2024-04-25T11:38:26.674198+0000","last_fullsized":"2024-04-25T11:38:26.674198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763520+0000","last_change":"2024-04-25T11:38:25.683663+0000","last_active":"2024-04-25T11:38:26.763520+0000","last_peered":"2024-04-25T11:38:26.763520+0000","last_clean":"2024-04-25T11:38:26.763520+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:38:26.763520+0000","last_undegraded":"2024-04-25T11:38:26.763520+0000","last_fullsized":"2024-04-25T11:38:26.763520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.049503+0000","last_change":"2024-04-25T11:38:24.670671+0000","last_active":"2024-04-25T11:38:53.049503+0000","last_peered":"2024-04-25T11:38:53.049503+0000","last_clean":"2024-04-25T11:38:53.049503+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:38:53.049503+0000","last_undegraded":"2024-04-25T11:38:53.049503+0000","last_fullsized":"2024-04-25T11:38:53.049503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.763017+0000","last_change":"2024-04-25T11:38:25.681439+0000","last_active":"2024-04-25T11:38:26.763017+0000","last_peered":"2024-04-25T11:38:26.763017+0000","last_clean":"2024-04-25T11:38:26.763017+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:38:26.763017+0000","last_undegraded":"2024-04-25T11:38:26.763017+0000","last_fullsized":"2024-04-25T11:38:26.763017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.050209+0000","last_change":"2024-04-25T11:38:24.669485+0000","last_active":"2024-04-25T11:38:53.050209+0000","last_peered":"2024-04-25T11:38:53.050209+0000","last_clean":"2024-04-25T11:38:53.050209+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:38:53.050209+0000","last_undegraded":"2024-04-25T11:38:53.050209+0000","last_fullsized":"2024-04-25T11:38:53.050209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761766+0000","last_change":"2024-04-25T11:38:25.684415+0000","last_active":"2024-04-25T11:38:26.761766+0000","last_peered":"2024-04-25T11:38:26.761766+0000","last_clean":"2024-04-25T11:38:26.761766+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:38:26.761766+0000","last_undegraded":"2024-04-25T11:38:26.761766+0000","last_fullsized":"2024-04-25T11:38:26.761766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:30.678908+0000","last_change":"2024-04-25T11:38:24.658741+0000","last_active":"2024-04-25T11:38:30.678908+0000","last_peered":"2024-04-25T11:38:30.678908+0000","last_clean":"2024-04-25T11:38:30.678908+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:38:30.678908+0000","last_undegraded":"2024-04-25T11:38:30.678908+0000","last_fullsized":"2024-04-25T11:38:30.678908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:26.761168+0000","last_change":"2024-04-25T11:38:25.667505+0000","last_active":"2024-04-25T11:38:26.761168+0000","last_peered":"2024-04-25T11:38:26.761168+0000","last_clean":"2024-04-25T11:38:26.761168+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:38:26.761168+0000","last_undegraded":"2024-04-25T11:38:26.761168+0000","last_fullsized":"2024-04-25T11:38:26.761168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T11:38:53.049060+0000","last_change":"2024-04-25T11:38:24.669215+0000","last_active":"2024-04-25T11:38:53.049060+0000","last_peered":"2024-04-25T11:38:53.049060+0000","last_clean":"2024-04-25T11:38:53.049060+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:38:53.049060+0000","last_undegraded":"2024-04-25T11:38:53.049060+0000","last_fullsized":"2024-04-25T11:38:53.049060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11177,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":218,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":22354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":663984,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":75315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":75448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":75349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":662176,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:39:13.035 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:39:13.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.189+0000 7fafddfe5640 1 Processor -- start 2024-04-25T11:39:13.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.189+0000 7fafddfe5640 1 -- start start 2024-04-25T11:39:13.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.189+0000 7fafddfe5640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd8155160 0x7fafd815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.189+0000 7fafddfe5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafd80c7c00 0x7fafd815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.189+0000 7fafddfe5640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fafd80cd840 0x7fafd8070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.189+0000 7fafddfe5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fafd815e270 con 0x7fafd80c7c00 2024-04-25T11:39:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.189+0000 7fafddfe5640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fafd815f200 con 0x7fafd80cd840 2024-04-25T11:39:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.189+0000 7fafddfe5640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fafd815b4e0 con 0x7fafd8155160 2024-04-25T11:39:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafd80c7c00 0x7fafd815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafd80c7c00 0x7fafd815aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37590/0 (socket says 172.21.15.40:37590) 2024-04-25T11:39:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd6ffd640 1 -- 172.21.15.40:0/955793336 learned_addr learned my addr 172.21.15.40:0/955793336 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd77fe640 1 --2- 172.21.15.40:0/955793336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd8155160 0x7fafd815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd7fff640 1 --2- 172.21.15.40:0/955793336 >> v2:172.21.15.62:3300/0 conn(0x7fafd80cd840 0x7fafd8070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd6ffd640 1 -- 172.21.15.40:0/955793336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd8155160 msgr2=0x7fafd815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/955793336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd8155160 0x7fafd815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd6ffd640 1 -- 172.21.15.40:0/955793336 >> v2:172.21.15.62:3300/0 conn(0x7fafd80cd840 msgr2=0x7fafd8070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/955793336 >> v2:172.21.15.62:3300/0 conn(0x7fafd80cd840 0x7fafd8070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd6ffd640 1 -- 172.21.15.40:0/955793336 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd815b7c0 con 0x7fafd80c7c00 2024-04-25T11:39:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.190+0000 7fafd7fff640 1 --2- 172.21.15.40:0/955793336 >> v2:172.21.15.62:3300/0 conn(0x7fafd80cd840 0x7fafd8070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/955793336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80c7c00 0x7fafd815aca0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fafd8075a90 tx=0x7fafcc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a156547fd058868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafd67fc640 1 -- 172.21.15.40:0/955793336 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc004030 con 0x7fafd80c7c00 2024-04-25T11:39:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafd67fc640 1 -- 172.21.15.40:0/955793336 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafcc0041d0 con 0x7fafd80c7c00 2024-04-25T11:39:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafd67fc640 1 -- 172.21.15.40:0/955793336 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc003610 con 0x7fafd80c7c00 2024-04-25T11:39:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafddfe5640 1 -- 172.21.15.40:0/955793336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80c7c00 msgr2=0x7fafd815aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafddfe5640 1 --2- 172.21.15.40:0/955793336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80c7c00 0x7fafd815aca0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fafd8075a90 tx=0x7fafcc02a250 comp rx=0 tx=0).stop 2024-04-25T11:39:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafddfe5640 1 -- 172.21.15.40:0/955793336 shutdown_connections 2024-04-25T11:39:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafddfe5640 1 --2- 172.21.15.40:0/955793336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80c7c00 0x7fafd815aca0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafddfe5640 1 --2- 172.21.15.40:0/955793336 >> v2:172.21.15.62:3300/0 conn(0x7fafd80cd840 0x7fafd8070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafddfe5640 1 --2- 172.21.15.40:0/955793336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd8155160 0x7fafd815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafddfe5640 1 -- 172.21.15.40:0/955793336 >> 172.21.15.40:0/955793336 conn(0x7fafd8072750 msgr2=0x7fafd8072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.191+0000 7fafddfe5640 1 -- 172.21.15.40:0/955793336 shutdown_connections 2024-04-25T11:39:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.192+0000 7fafddfe5640 1 -- 172.21.15.40:0/955793336 wait complete. 2024-04-25T11:39:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.193+0000 7fafddfe5640 1 Processor -- start 2024-04-25T11:39:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.193+0000 7fafddfe5640 1 -- start start 2024-04-25T11:39:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafddfe5640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd80c7c00 0x7fafd81479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafddfe5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80cd840 0x7fafd81f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafddfe5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8155160 0x7fafd80c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafddfe5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafd815e7a0 con 0x7fafd80cd840 2024-04-25T11:39:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafddfe5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fafd815e4c0 con 0x7fafd8155160 2024-04-25T11:39:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafddfe5640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fafd815b1e0 con 0x7fafd80c7c00 2024-04-25T11:39:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80cd840 0x7fafd81f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80cd840 0x7fafd81f12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37600/0 (socket says 172.21.15.40:37600) 2024-04-25T11:39:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd6ffd640 1 -- 172.21.15.40:0/3961120200 learned_addr learned my addr 172.21.15.40:0/3961120200 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd77fe640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd80c7c00 0x7fafd81479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd7fff640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8155160 0x7fafd80c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd6ffd640 1 -- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd80c7c00 msgr2=0x7fafd81479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd80c7c00 0x7fafd81479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd6ffd640 1 -- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8155160 msgr2=0x7fafd80c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8155160 0x7fafd80c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd6ffd640 1 -- 172.21.15.40:0/3961120200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd80c53c0 con 0x7fafd80cd840 2024-04-25T11:39:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafd6ffd640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80cd840 0x7fafd81f12f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fafcc02a290 tx=0x7fafcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafbffff640 1 -- 172.21.15.40:0/3961120200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc041020 con 0x7fafd80cd840 2024-04-25T11:39:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafbffff640 1 -- 172.21.15.40:0/3961120200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafcc003c50 con 0x7fafd80cd840 2024-04-25T11:39:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd80c5640 con 0x7fafd80cd840 2024-04-25T11:39:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.194+0000 7fafbffff640 1 -- 172.21.15.40:0/3961120200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc004030 con 0x7fafd80cd840 2024-04-25T11:39:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.195+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd8159470 con 0x7fafd80cd840 2024-04-25T11:39:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.195+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd8069200 con 0x7fafd80cd840 2024-04-25T11:39:13.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.197+0000 7fafbffff640 1 -- 172.21.15.40:0/3961120200 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafcc044020 con 0x7fafd80cd840 2024-04-25T11:39:13.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.197+0000 7fafbffff640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fafc406c920 0x7fafc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.197+0000 7fafbffff640 1 -- 172.21.15.40:0/3961120200 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7fafcc0ac6b0 con 0x7fafd80cd840 2024-04-25T11:39:13.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.197+0000 7fafd77fe640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fafc406c920 0x7fafc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.198+0000 7fafd77fe640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fafc406c920 0x7fafc406ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fafc8011340 tx=0x7fafc800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:13.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.206+0000 7fafbffff640 1 -- 172.21.15.40:0/3961120200 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fafcc04b030 con 0x7fafd80cd840 2024-04-25T11:39:13.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.393+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafd8148490 con 0x7fafd80cd840 2024-04-25T11:39:13.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.394+0000 7fafbffff640 1 -- 172.21.15.40:0/3961120200 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16806 (secure 0 0 0) 0x7fafd80c53c0 con 0x7fafd80cd840 2024-04-25T11:39:13.382 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:13.382 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:39:13.102277+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:39:13.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fafc406c920 msgr2=0x7fafc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fafc406c920 0x7fafc406ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fafc8011340 tx=0x7fafc800b040 comp rx=0 tx=0).stop 2024-04-25T11:39:13.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80cd840 msgr2=0x7fafd81f12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80cd840 0x7fafd81f12f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fafcc02a290 tx=0x7fafcc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 shutdown_connections 2024-04-25T11:39:13.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fafc406c920 0x7fafc406ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafd8155160 0x7fafd80c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80cd840 0x7fafd81f12f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 --2- 172.21.15.40:0/3961120200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafd80c7c00 0x7fafd81479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 >> 172.21.15.40:0/3961120200 conn(0x7fafd8072750 msgr2=0x7fafd80cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:13.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 shutdown_connections 2024-04-25T11:39:13.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:13.397+0000 7fafddfe5640 1 -- 172.21.15.40:0/3961120200 wait complete. 2024-04-25T11:39:14.397 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T11:39:14.397 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:39:14.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.516+0000 7f7a87e4e640 1 Processor -- start 2024-04-25T11:39:14.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.516+0000 7f7a87e4e640 1 -- start start 2024-04-25T11:39:14.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a87e4e640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7a800c2730 0x7f7a800c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a87e4e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a80067b20 0x7f7a8015b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a87e4e640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8015bbc0 con 0x7f7a800c7670 2024-04-25T11:39:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a87e4e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7a800c8580 con 0x7f7a80067b20 2024-04-25T11:39:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a87e4e640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f7a80155380 con 0x7f7a800c2730 2024-04-25T11:39:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a853e9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a80067b20 0x7f7a8015b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a853e9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a80067b20 0x7f7a8015b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37602/0 (socket says 172.21.15.40:37602) 2024-04-25T11:39:14.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a853e9640 1 -- 172.21.15.40:0/1942846707 learned_addr learned my addr 172.21.15.40:0/1942846707 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:14.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a85bea640 1 --2- 172.21.15.40:0/1942846707 >> v2:172.21.15.62:3300/0 conn(0x7f7a800c2730 0x7f7a800c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a853e9640 1 -- 172.21.15.40:0/1942846707 >> v2:172.21.15.62:3300/0 conn(0x7f7a800c2730 msgr2=0x7f7a800c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:14.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a853e9640 1 --2- 172.21.15.40:0/1942846707 >> v2:172.21.15.62:3300/0 conn(0x7f7a800c2730 0x7f7a800c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.517+0000 7f7a853e9640 1 -- 172.21.15.40:0/1942846707 >> v1:172.21.15.62:6789/0 conn(0x7f7a800c7670 legacy=0x7f7a800c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:14.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.518+0000 7f7a853e9640 1 -- 172.21.15.40:0/1942846707 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a80072b20 con 0x7f7a80067b20 2024-04-25T11:39:14.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.518+0000 7f7a853e9640 1 --2- 172.21.15.40:0/1942846707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a80067b20 0x7f7a8015b680 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a80070880 tx=0x7f7a7402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162f042578b6bbbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.518+0000 7f7a84be8640 1 -- 172.21.15.40:0/1942846707 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74037070 con 0x7f7a80067b20 2024-04-25T11:39:14.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.518+0000 7f7a84be8640 1 -- 172.21.15.40:0/1942846707 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a74032440 con 0x7f7a80067b20 2024-04-25T11:39:14.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.518+0000 7f7a84be8640 1 -- 172.21.15.40:0/1942846707 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74032810 con 0x7f7a80067b20 2024-04-25T11:39:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.519+0000 7f7a87e4e640 1 -- 172.21.15.40:0/1942846707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a80067b20 msgr2=0x7f7a8015b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.519+0000 7f7a87e4e640 1 --2- 172.21.15.40:0/1942846707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a80067b20 0x7f7a8015b680 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a80070880 tx=0x7f7a7402a230 comp rx=0 tx=0).stop 2024-04-25T11:39:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.519+0000 7f7a87e4e640 1 -- 172.21.15.40:0/1942846707 shutdown_connections 2024-04-25T11:39:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.519+0000 7f7a87e4e640 1 --2- 172.21.15.40:0/1942846707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a80067b20 0x7f7a8015b680 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.519+0000 7f7a87e4e640 1 --2- 172.21.15.40:0/1942846707 >> v2:172.21.15.62:3300/0 conn(0x7f7a800c2730 0x7f7a800c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.519+0000 7f7a87e4e640 1 -- 172.21.15.40:0/1942846707 >> 172.21.15.40:0/1942846707 conn(0x7f7a800c9650 msgr2=0x7f7a800c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.519+0000 7f7a87e4e640 1 -- 172.21.15.40:0/1942846707 shutdown_connections 2024-04-25T11:39:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.519+0000 7f7a87e4e640 1 -- 172.21.15.40:0/1942846707 wait complete. 2024-04-25T11:39:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.520+0000 7f7a87e4e640 1 Processor -- start 2024-04-25T11:39:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.520+0000 7f7a87e4e640 1 -- start start 2024-04-25T11:39:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a87e4e640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a800c2730 0x7f7a801f1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a87e4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800c7670 0x7f7a80152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a87e4e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a8014ee20 0x7f7a8014f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a87e4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800730a0 con 0x7f7a800c7670 2024-04-25T11:39:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a87e4e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800739d0 con 0x7f7a8014ee20 2024-04-25T11:39:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a87e4e640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7a80073380 con 0x7f7a800c2730 2024-04-25T11:39:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a853e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800c7670 0x7f7a80152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a853e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800c7670 0x7f7a80152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37604/0 (socket says 172.21.15.40:37604) 2024-04-25T11:39:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a853e9640 1 -- 172.21.15.40:0/992655195 learned_addr learned my addr 172.21.15.40:0/992655195 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a853e9640 1 -- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a800c2730 msgr2=0x7f7a801f1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a853e9640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a800c2730 0x7f7a801f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a853e9640 1 -- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a8014ee20 msgr2=0x7f7a8014f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a863eb640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a8014ee20 0x7f7a8014f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a853e9640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a8014ee20 0x7f7a8014f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a853e9640 1 -- 172.21.15.40:0/992655195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8014f7f0 con 0x7f7a800c7670 2024-04-25T11:39:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.521+0000 7f7a863eb640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a8014ee20 0x7f7a8014f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.522+0000 7f7a853e9640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800c7670 0x7f7a80152e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a74000c00 tx=0x7f7a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.522+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/992655195 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74041020 con 0x7f7a800c7670 2024-04-25T11:39:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.522+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/992655195 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a740306c0 con 0x7f7a800c7670 2024-04-25T11:39:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.522+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a801512e0 con 0x7f7a800c7670 2024-04-25T11:39:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.522+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/992655195 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74037040 con 0x7f7a800c7670 2024-04-25T11:39:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.522+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a800c4360 con 0x7f7a800c7670 2024-04-25T11:39:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.523+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a801515c0 con 0x7f7a800c7670 2024-04-25T11:39:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.523+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/992655195 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a74044020 con 0x7f7a800c7670 2024-04-25T11:39:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.524+0000 7f7a7e7fc640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7a6c06c8f0 0x7f7a6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.524+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/992655195 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7a740ac3e0 con 0x7f7a800c7670 2024-04-25T11:39:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.524+0000 7f7a85bea640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7a6c06c8f0 0x7f7a6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.524+0000 7f7a7e7fc640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f7a6c074c70 0x7f7a6c0770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.524+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/992655195 --> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a7402be50 con 0x7f7a6c074c70 2024-04-25T11:39:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.524+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/992655195 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7a740b3050 con 0x7f7a800c7670 2024-04-25T11:39:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.524+0000 7f7a863eb640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f7a6c074c70 0x7f7a6c0770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.524+0000 7f7a863eb640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f7a6c074c70 0x7f7a6c0770f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.524+0000 7f7a85bea640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7a6c06c8f0 0x7f7a6c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a780098f0 tx=0x7f7a78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.525+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/992655195 <== osd.0 v2:172.21.15.40:6816/2617970798 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a7402be50 con 0x7f7a6c074c70 2024-04-25T11:39:14.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.538+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 --> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7a8014ebe0 con 0x7f7a6c074c70 2024-04-25T11:39:14.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.538+0000 7f7a7e7fc640 1 -- 172.21.15.40:0/992655195 <== osd.0 v2:172.21.15.40:6816/2617970798 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7a8014ebe0 con 0x7f7a6c074c70 2024-04-25T11:39:14.526 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:39:14.526 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:39:14.526 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:39:14.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.538+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f7a6c074c70 msgr2=0x7f7a6c0770f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.538+0000 7f7a87e4e640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.40:6816/2617970798,v1:172.21.15.40:6817/2617970798] conn(0x7f7a6c074c70 0x7f7a6c0770f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.539+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7a6c06c8f0 msgr2=0x7f7a6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.539+0000 7f7a87e4e640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7a6c06c8f0 0x7f7a6c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a780098f0 tx=0x7f7a78008040 comp rx=0 tx=0).stop 2024-04-25T11:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.539+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800c7670 msgr2=0x7f7a80152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.539+0000 7f7a87e4e640 1 --2- 172.21.15.40:0/992655195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800c7670 0x7f7a80152e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7a74000c00 tx=0x7f7a740046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.539+0000 7f7a863eb640 1 -- 172.21.15.40:0/992655195 reap_dead start 2024-04-25T11:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.539+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 shutdown_connections 2024-04-25T11:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.539+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 >> 172.21.15.40:0/992655195 conn(0x7f7a800c9650 msgr2=0x7f7a80156320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.539+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 shutdown_connections 2024-04-25T11:39:14.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.539+0000 7f7a87e4e640 1 -- 172.21.15.40:0/992655195 wait complete. 2024-04-25T11:39:14.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T11:39:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.651+0000 7f03b45fe640 1 Processor -- start 2024-04-25T11:39:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.651+0000 7f03b45fe640 1 -- start start 2024-04-25T11:39:14.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.652+0000 7f03b45fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f03ac0b6210 0x7f03ac0c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.652+0000 7f03b45fe640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03ac15f060 con 0x7f03ac154f50 2024-04-25T11:39:14.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.652+0000 7f03b45fe640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03ac0d1040 con 0x7f03ac0c8690 2024-04-25T11:39:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.652+0000 7f03b45fe640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f03ac15eb20 con 0x7f03ac0b6210 2024-04-25T11:39:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.652+0000 7f03b239a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f03ac154f50 0x7f03ac0cb2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34528/0 (socket says 172.21.15.40:34528) 2024-04-25T11:39:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.652+0000 7f03b239a640 1 -- 172.21.15.40:0/4112089653 learned_addr learned my addr 172.21.15.40:0/4112089653 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.653+0000 7f03b1b99640 1 --2- 172.21.15.40:0/4112089653 >> v2:172.21.15.62:3300/0 conn(0x7f03ac0b6210 0x7f03ac0c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.653+0000 7f03b1398640 1 -- 172.21.15.40:0/4112089653 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3171198071 0 0) 0x7f03ac15f060 con 0x7f03ac154f50 2024-04-25T11:39:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.653+0000 7f03b1398640 1 -- 172.21.15.40:0/4112089653 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0394003520 con 0x7f03ac154f50 2024-04-25T11:39:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.653+0000 7f03b1398640 1 -- 172.21.15.40:0/4112089653 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2425866856 0 0) 0x7f0394003520 con 0x7f03ac154f50 2024-04-25T11:39:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.653+0000 7f03b1b99640 1 -- 172.21.15.40:0/4112089653 >> v1:172.21.15.62:6789/0 conn(0x7f03ac0c8690 legacy=0x7f03ac0c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.653+0000 7f03b1b99640 1 -- 172.21.15.40:0/4112089653 >> v1:172.21.15.40:6789/0 conn(0x7f03ac154f50 legacy=0x7f03ac0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.653+0000 7f03b1b99640 1 -- 172.21.15.40:0/4112089653 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac0cc150 con 0x7f03ac0b6210 2024-04-25T11:39:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.654+0000 7f03b1b99640 1 --2- 172.21.15.40:0/4112089653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ac0b6210 0x7f03ac0c7a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0cc320 tx=0x7f03a002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e66667bd00d50b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.654+0000 7f03b1398640 1 -- 172.21.15.40:0/4112089653 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0037070 con 0x7f03ac0b6210 2024-04-25T11:39:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.654+0000 7f03b1398640 1 -- 172.21.15.40:0/4112089653 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a002aba0 con 0x7f03ac0b6210 2024-04-25T11:39:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.654+0000 7f03b1398640 1 -- 172.21.15.40:0/4112089653 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c0019f0 con 0x7f03ac0b6210 2024-04-25T11:39:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.654+0000 7f03b45fe640 1 -- 172.21.15.40:0/4112089653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ac0b6210 msgr2=0x7f03ac0c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.654+0000 7f03b45fe640 1 --2- 172.21.15.40:0/4112089653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ac0b6210 0x7f03ac0c7a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0cc320 tx=0x7f03a002a110 comp rx=0 tx=0).stop 2024-04-25T11:39:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.655+0000 7f03b45fe640 1 -- 172.21.15.40:0/4112089653 shutdown_connections 2024-04-25T11:39:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.655+0000 7f03b45fe640 1 --2- 172.21.15.40:0/4112089653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ac0b6210 0x7f03ac0c7a70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.655+0000 7f03b45fe640 1 -- 172.21.15.40:0/4112089653 >> 172.21.15.40:0/4112089653 conn(0x7f03ac0d0440 msgr2=0x7f03ac0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.655+0000 7f03b45fe640 1 -- 172.21.15.40:0/4112089653 shutdown_connections 2024-04-25T11:39:14.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.655+0000 7f03b45fe640 1 -- 172.21.15.40:0/4112089653 wait complete. 2024-04-25T11:39:14.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.656+0000 7f03b45fe640 1 Processor -- start 2024-04-25T11:39:14.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.656+0000 7f03b45fe640 1 -- start start 2024-04-25T11:39:14.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.656+0000 7f03b45fe640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03ac0c8690 0x7f03ac157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.656+0000 7f03b45fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ac154f50 0x7f03ac15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.656+0000 7f03b45fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ac15ba80 0x7f03ac15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.656+0000 7f03b45fe640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac0cc3f0 con 0x7f03ac154f50 2024-04-25T11:39:14.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.656+0000 7f03b45fe640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac15ed80 con 0x7f03ac15ba80 2024-04-25T11:39:14.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.656+0000 7f03b45fe640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f03ac0c8170 con 0x7f03ac0c8690 2024-04-25T11:39:14.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.656+0000 7f03b1b99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ac154f50 0x7f03ac15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b1b99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ac154f50 0x7f03ac15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37612/0 (socket says 172.21.15.40:37612) 2024-04-25T11:39:14.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b1b99640 1 -- 172.21.15.40:0/225667908 learned_addr learned my addr 172.21.15.40:0/225667908 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:14.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b239a640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03ac0c8690 0x7f03ac157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b2b9b640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ac15ba80 0x7f03ac15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b1b99640 1 -- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03ac0c8690 msgr2=0x7f03ac157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b1b99640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03ac0c8690 0x7f03ac157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b1b99640 1 -- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ac15ba80 msgr2=0x7f03ac15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b1b99640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ac15ba80 0x7f03ac15ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b1b99640 1 -- 172.21.15.40:0/225667908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac1f3c00 con 0x7f03ac154f50 2024-04-25T11:39:14.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b1b99640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ac154f50 0x7f03ac15a5c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009890 tx=0x7f03a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03aaffd640 1 -- 172.21.15.40:0/225667908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0042020 con 0x7f03ac154f50 2024-04-25T11:39:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03aaffd640 1 -- 172.21.15.40:0/225667908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a003d670 con 0x7f03ac154f50 2024-04-25T11:39:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac1f0780 con 0x7f03ac154f50 2024-04-25T11:39:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03aaffd640 1 -- 172.21.15.40:0/225667908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0037040 con 0x7f03ac154f50 2024-04-25T11:39:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.657+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1f0d40 con 0x7f03ac154f50 2024-04-25T11:39:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.658+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac04fd50 con 0x7f03ac154f50 2024-04-25T11:39:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.659+0000 7f03aaffd640 1 -- 172.21.15.40:0/225667908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03a0045020 con 0x7f03ac154f50 2024-04-25T11:39:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.659+0000 7f03aaffd640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f039806c8f0 0x7f039806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.659+0000 7f03aaffd640 1 -- 172.21.15.40:0/225667908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f03a00ac110 con 0x7f03ac154f50 2024-04-25T11:39:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.659+0000 7f03b239a640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f039806c8f0 0x7f039806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.660+0000 7f03b239a640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f039806c8f0 0x7f039806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f039c005c00 tx=0x7f039c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.664+0000 7f03aaffd640 1 -- 172.21.15.40:0/225667908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03a004c070 con 0x7f03ac154f50 2024-04-25T11:39:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.847+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f03ac1f39a0 con 0x7f039806c8f0 2024-04-25T11:39:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.847+0000 7f03aaffd640 1 -- 172.21.15.40:0/225667908 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f039c014450 con 0x7f039806c8f0 2024-04-25T11:39:14.834 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T11:39:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.849+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f039806c8f0 msgr2=0x7f039806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.849+0000 7f03b45fe640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f039806c8f0 0x7f039806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f039c005c00 tx=0x7f039c009040 comp rx=0 tx=0).stop 2024-04-25T11:39:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ac154f50 msgr2=0x7f03ac15a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ac154f50 0x7f03ac15a5c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009890 tx=0x7f03a00046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 shutdown_connections 2024-04-25T11:39:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f039806c8f0 0x7f039806edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ac15ba80 0x7f03ac15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03ac154f50 0x7f03ac15a5c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 --2- 172.21.15.40:0/225667908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03ac0c8690 0x7f03ac157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 >> 172.21.15.40:0/225667908 conn(0x7f03ac0d0440 msgr2=0x7f03ac152720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 shutdown_connections 2024-04-25T11:39:14.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.850+0000 7f03b45fe640 1 -- 172.21.15.40:0/225667908 wait complete. 2024-04-25T11:39:14.847 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T11:39:14.847 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:39:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.966+0000 7f5449a43640 1 Processor -- start 2024-04-25T11:39:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.966+0000 7f5449a43640 1 -- start start 2024-04-25T11:39:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f5449a43640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5444158440 0x7f54440ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f5449a43640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f544406ca80 0x7f5444071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f5449a43640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5444071da0 0x7f544415b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f5449a43640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f54440c9f90 con 0x7f5444071da0 2024-04-25T11:39:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f5449a43640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f54440caea0 con 0x7f544406ca80 2024-04-25T11:39:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f5449a43640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f54440c9050 con 0x7f5444158440 2024-04-25T11:39:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f54437fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5444071da0 0x7f544415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f54437fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5444071da0 0x7f544415b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37628/0 (socket says 172.21.15.40:37628) 2024-04-25T11:39:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f54437fe640 1 -- 172.21.15.40:0/648797888 learned_addr learned my addr 172.21.15.40:0/648797888 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f543bfff640 1 --2- 172.21.15.40:0/648797888 >> v2:172.21.15.62:3300/0 conn(0x7f544406ca80 0x7f5444071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f5442ffd640 1 --2- 172.21.15.40:0/648797888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5444158440 0x7f54440ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f54437fe640 1 -- 172.21.15.40:0/648797888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5444158440 msgr2=0x7f54440ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f54437fe640 1 --2- 172.21.15.40:0/648797888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5444158440 0x7f54440ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f54437fe640 1 -- 172.21.15.40:0/648797888 >> v2:172.21.15.62:3300/0 conn(0x7f544406ca80 msgr2=0x7f5444071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f54437fe640 1 --2- 172.21.15.40:0/648797888 >> v2:172.21.15.62:3300/0 conn(0x7f544406ca80 0x7f5444071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.967+0000 7f54437fe640 1 -- 172.21.15.40:0/648797888 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54440c9330 con 0x7f5444071da0 2024-04-25T11:39:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.968+0000 7f54437fe640 1 --2- 172.21.15.40:0/648797888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5444071da0 0x7f544415b310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f54440cf5d0 tx=0x7f543c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599f387ae1eea715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.968+0000 7f54427fc640 1 -- 172.21.15.40:0/648797888 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c036070 con 0x7f5444071da0 2024-04-25T11:39:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.968+0000 7f54427fc640 1 -- 172.21.15.40:0/648797888 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543c031600 con 0x7f5444071da0 2024-04-25T11:39:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.968+0000 7f54427fc640 1 -- 172.21.15.40:0/648797888 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0319d0 con 0x7f5444071da0 2024-04-25T11:39:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.969+0000 7f5449a43640 1 -- 172.21.15.40:0/648797888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5444071da0 msgr2=0x7f544415b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.969+0000 7f5449a43640 1 --2- 172.21.15.40:0/648797888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5444071da0 0x7f544415b310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f54440cf5d0 tx=0x7f543c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.969+0000 7f5449a43640 1 -- 172.21.15.40:0/648797888 shutdown_connections 2024-04-25T11:39:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.969+0000 7f5449a43640 1 --2- 172.21.15.40:0/648797888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5444071da0 0x7f544415b310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.969+0000 7f5449a43640 1 --2- 172.21.15.40:0/648797888 >> v2:172.21.15.62:3300/0 conn(0x7f544406ca80 0x7f5444071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.969+0000 7f5449a43640 1 --2- 172.21.15.40:0/648797888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5444158440 0x7f54440ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.969+0000 7f5449a43640 1 -- 172.21.15.40:0/648797888 >> 172.21.15.40:0/648797888 conn(0x7f5444070600 msgr2=0x7f54440709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.969+0000 7f5449a43640 1 -- 172.21.15.40:0/648797888 shutdown_connections 2024-04-25T11:39:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.969+0000 7f5449a43640 1 -- 172.21.15.40:0/648797888 wait complete. 2024-04-25T11:39:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.970+0000 7f5449a43640 1 Processor -- start 2024-04-25T11:39:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.970+0000 7f5449a43640 1 -- start start 2024-04-25T11:39:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5449a43640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f544406ca80 0x7f54440b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5449a43640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5444071da0 0x7f54440b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5449a43640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5444158440 0x7f54440afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5449a43640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5444158820 con 0x7f544406ca80 2024-04-25T11:39:14.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5449a43640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f54440ca4c0 con 0x7f5444158440 2024-04-25T11:39:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5449a43640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f54441589a0 con 0x7f5444071da0 2024-04-25T11:39:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f544406ca80 0x7f54440b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f544406ca80 0x7f54440b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37640/0 (socket says 172.21.15.40:37640) 2024-04-25T11:39:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5442ffd640 1 -- 172.21.15.40:0/3105458421 learned_addr learned my addr 172.21.15.40:0/3105458421 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f54437fe640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5444158440 0x7f54440afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f543bfff640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5444071da0 0x7f54440b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5442ffd640 1 -- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5444071da0 msgr2=0x7f54440b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5442ffd640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5444071da0 0x7f54440b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5442ffd640 1 -- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5444158440 msgr2=0x7f54440afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5442ffd640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5444158440 0x7f54440afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5442ffd640 1 -- 172.21.15.40:0/3105458421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544414fb60 con 0x7f544406ca80 2024-04-25T11:39:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.971+0000 7f5442ffd640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f544406ca80 0x7f54440b3fc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f543400e8a0 tx=0x7f543400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.972+0000 7f5440ff9640 1 -- 172.21.15.40:0/3105458421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434017070 con 0x7f544406ca80 2024-04-25T11:39:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.972+0000 7f5440ff9640 1 -- 172.21.15.40:0/3105458421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543400c8e0 con 0x7f544406ca80 2024-04-25T11:39:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.972+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544414f2c0 con 0x7f544406ca80 2024-04-25T11:39:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.972+0000 7f5440ff9640 1 -- 172.21.15.40:0/3105458421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543400ccd0 con 0x7f544406ca80 2024-04-25T11:39:14.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.972+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544414f880 con 0x7f544406ca80 2024-04-25T11:39:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.973+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5444068bb0 con 0x7f544406ca80 2024-04-25T11:39:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.973+0000 7f5440ff9640 1 -- 172.21.15.40:0/3105458421 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f543401a070 con 0x7f544406ca80 2024-04-25T11:39:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.974+0000 7f5440ff9640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f542406c8f0 0x7f542406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.974+0000 7f5440ff9640 1 -- 172.21.15.40:0/3105458421 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f543408bce0 con 0x7f544406ca80 2024-04-25T11:39:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.974+0000 7f5440ff9640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7f5424074c10 0x7f5424077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.974+0000 7f543bfff640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f542406c8f0 0x7f542406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.974+0000 7f5440ff9640 1 -- 172.21.15.40:0/3105458421 --> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5434016ec0 con 0x7f5424074c10 2024-04-25T11:39:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.974+0000 7f5440ff9640 1 -- 172.21.15.40:0/3105458421 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f543408e3e0 con 0x7f544406ca80 2024-04-25T11:39:14.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.974+0000 7f54437fe640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7f5424074c10 0x7f5424077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.974+0000 7f543bfff640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f542406c8f0 0x7f542406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f54441f1ca0 tx=0x7f542c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.974+0000 7f54437fe640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7f5424074c10 0x7f5424077090 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.975+0000 7f5440ff9640 1 -- 172.21.15.40:0/3105458421 <== osd.1 v2:172.21.15.40:6800/3790556570 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5434016ec0 con 0x7f5424074c10 2024-04-25T11:39:14.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.989+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 --> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f54440b4580 con 0x7f5424074c10 2024-04-25T11:39:14.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.989+0000 7f5440ff9640 1 -- 172.21.15.40:0/3105458421 <== osd.1 v2:172.21.15.40:6800/3790556570 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f54440b4580 con 0x7f5424074c10 2024-04-25T11:39:14.977 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:39:14.977 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:39:14.977 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:39:14.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7f5424074c10 msgr2=0x7f5424077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.40:6800/3790556570,v1:172.21.15.40:6801/3790556570] conn(0x7f5424074c10 0x7f5424077090 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f542406c8f0 msgr2=0x7f542406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f542406c8f0 0x7f542406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f54441f1ca0 tx=0x7f542c009190 comp rx=0 tx=0).stop 2024-04-25T11:39:14.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f544406ca80 msgr2=0x7f54440b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 --2- 172.21.15.40:0/3105458421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f544406ca80 0x7f54440b3fc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f543400e8a0 tx=0x7f543400ed60 comp rx=0 tx=0).stop 2024-04-25T11:39:14.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f54437fe640 1 -- 172.21.15.40:0/3105458421 reap_dead start 2024-04-25T11:39:14.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 shutdown_connections 2024-04-25T11:39:14.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 >> 172.21.15.40:0/3105458421 conn(0x7f5444070600 msgr2=0x7f5444159350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 shutdown_connections 2024-04-25T11:39:14.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:14.990+0000 7f5449a43640 1 -- 172.21.15.40:0/3105458421 wait complete. 2024-04-25T11:39:14.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T11:39:15.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.101+0000 7fed4fe10640 1 Processor -- start 2024-04-25T11:39:15.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.102+0000 7fed4fe10640 1 -- start start 2024-04-25T11:39:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.102+0000 7fed4fe10640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed48154ff0 0x7fed4815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.102+0000 7fed4fe10640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed480b3fc0 0x7fed480cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.102+0000 7fed4fe10640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed480cbfc0 con 0x7fed4815b9e0 2024-04-25T11:39:15.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.102+0000 7fed4fe10640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fed480cc280 con 0x7fed48154ff0 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.102+0000 7fed4fe10640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fed480cc560 con 0x7fed480b3fc0 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4dbac640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed48154ff0 0x7fed4815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4dbac640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed48154ff0 0x7fed4815b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37652/0 (socket says 172.21.15.40:37652) 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4e3ad640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fed4815b9e0 0x7fed480d04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34532/0 (socket says 172.21.15.40:34532) 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4dbac640 1 -- 172.21.15.40:0/913056573 learned_addr learned my addr 172.21.15.40:0/913056573 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4d3ab640 1 --2- 172.21.15.40:0/913056573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed480b3fc0 0x7fed480cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4dbac640 1 -- 172.21.15.40:0/913056573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed480b3fc0 msgr2=0x7fed480cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4dbac640 1 --2- 172.21.15.40:0/913056573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed480b3fc0 0x7fed480cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4dbac640 1 -- 172.21.15.40:0/913056573 >> v1:172.21.15.40:6789/0 conn(0x7fed4815b9e0 legacy=0x7fed480d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4dbac640 1 -- 172.21.15.40:0/913056573 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed480cc840 con 0x7fed48154ff0 2024-04-25T11:39:15.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.103+0000 7fed4cbaa640 1 -- 172.21.15.40:0/913056573 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867244414 0 0) 0x7fed480cbfc0 con 0x7fed4815b9e0 2024-04-25T11:39:15.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4dbac640 1 --2- 172.21.15.40:0/913056573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed48154ff0 0x7fed4815b4a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed44009870 tx=0x7fed4402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80fe3bbf624660c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4cbaa640 1 -- 172.21.15.40:0/913056573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44037070 con 0x7fed48154ff0 2024-04-25T11:39:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4cbaa640 1 -- 172.21.15.40:0/913056573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed4402ae50 con 0x7fed48154ff0 2024-04-25T11:39:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4cbaa640 1 -- 172.21.15.40:0/913056573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44032860 con 0x7fed48154ff0 2024-04-25T11:39:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4fe10640 1 -- 172.21.15.40:0/913056573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed48154ff0 msgr2=0x7fed4815b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4fe10640 1 --2- 172.21.15.40:0/913056573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed48154ff0 0x7fed4815b4a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed44009870 tx=0x7fed4402a300 comp rx=0 tx=0).stop 2024-04-25T11:39:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4fe10640 1 -- 172.21.15.40:0/913056573 shutdown_connections 2024-04-25T11:39:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4fe10640 1 --2- 172.21.15.40:0/913056573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed480b3fc0 0x7fed480cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4fe10640 1 --2- 172.21.15.40:0/913056573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed48154ff0 0x7fed4815b4a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4fe10640 1 -- 172.21.15.40:0/913056573 >> 172.21.15.40:0/913056573 conn(0x7fed4806c680 msgr2=0x7fed4806ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4fe10640 1 -- 172.21.15.40:0/913056573 shutdown_connections 2024-04-25T11:39:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.104+0000 7fed4fe10640 1 -- 172.21.15.40:0/913056573 wait complete. 2024-04-25T11:39:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4fe10640 1 Processor -- start 2024-04-25T11:39:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4fe10640 1 -- start start 2024-04-25T11:39:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4fe10640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed480b3fc0 0x7fed4815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4fe10640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed48154ff0 0x7fed481489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4fe10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed4815b9e0 0x7fed48148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4fe10640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fed481553d0 con 0x7fed4815b9e0 2024-04-25T11:39:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4fe10640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed4815af80 con 0x7fed480b3fc0 2024-04-25T11:39:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4fe10640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fed4815f0c0 con 0x7fed48154ff0 2024-04-25T11:39:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4e3ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed4815b9e0 0x7fed48148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4e3ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed4815b9e0 0x7fed48148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37654/0 (socket says 172.21.15.40:37654) 2024-04-25T11:39:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.106+0000 7fed4e3ad640 1 -- 172.21.15.40:0/155174023 learned_addr learned my addr 172.21.15.40:0/155174023 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4dbac640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed480b3fc0 0x7fed4815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4d3ab640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed48154ff0 0x7fed481489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4e3ad640 1 -- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed48154ff0 msgr2=0x7fed481489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4e3ad640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed48154ff0 0x7fed481489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4e3ad640 1 -- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed480b3fc0 msgr2=0x7fed4815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4e3ad640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed480b3fc0 0x7fed4815c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4e3ad640 1 -- 172.21.15.40:0/155174023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed48152b10 con 0x7fed4815b9e0 2024-04-25T11:39:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4e3ad640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed4815b9e0 0x7fed48148f30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c700 tx=0x7fed4000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed3e7fc640 1 -- 172.21.15.40:0/155174023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40018070 con 0x7fed4815b9e0 2024-04-25T11:39:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed3e7fc640 1 -- 172.21.15.40:0/155174023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed4000ce10 con 0x7fed4815b9e0 2024-04-25T11:39:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed480c7a00 con 0x7fed4815b9e0 2024-04-25T11:39:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed3e7fc640 1 -- 172.21.15.40:0/155174023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4000a880 con 0x7fed4815b9e0 2024-04-25T11:39:15.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.107+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4815ede0 con 0x7fed4815b9e0 2024-04-25T11:39:15.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.108+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4804fd50 con 0x7fed4815b9e0 2024-04-25T11:39:15.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.109+0000 7fed3e7fc640 1 -- 172.21.15.40:0/155174023 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed4001b070 con 0x7fed4815b9e0 2024-04-25T11:39:15.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.109+0000 7fed3e7fc640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed2c06c8f0 0x7fed2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.109+0000 7fed3e7fc640 1 -- 172.21.15.40:0/155174023 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fed4008b0f0 con 0x7fed4815b9e0 2024-04-25T11:39:15.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.109+0000 7fed4dbac640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed2c06c8f0 0x7fed2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.110+0000 7fed4dbac640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed2c06c8f0 0x7fed2c06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fed44034000 tx=0x7fed4402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.113+0000 7fed3e7fc640 1 -- 172.21.15.40:0/155174023 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed400594a0 con 0x7fed4815b9e0 2024-04-25T11:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.299+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fed480cdc90 con 0x7fed2c06c8f0 2024-04-25T11:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.299+0000 7fed3e7fc640 1 -- 172.21.15.40:0/155174023 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fed480cdc90 con 0x7fed2c06c8f0 2024-04-25T11:39:15.286 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T11:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.301+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed2c06c8f0 msgr2=0x7fed2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.301+0000 7fed4fe10640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed2c06c8f0 0x7fed2c06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fed44034000 tx=0x7fed4402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.301+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed4815b9e0 msgr2=0x7fed48148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.301+0000 7fed4fe10640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed4815b9e0 0x7fed48148f30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c700 tx=0x7fed4000cbc0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.302+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 shutdown_connections 2024-04-25T11:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.302+0000 7fed4fe10640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fed2c06c8f0 0x7fed2c06edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.302+0000 7fed4fe10640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed4815b9e0 0x7fed48148f30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.302+0000 7fed4fe10640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed48154ff0 0x7fed481489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.302+0000 7fed4fe10640 1 --2- 172.21.15.40:0/155174023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed480b3fc0 0x7fed4815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.302+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 >> 172.21.15.40:0/155174023 conn(0x7fed4806c680 msgr2=0x7fed48153920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.302+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 shutdown_connections 2024-04-25T11:39:15.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.302+0000 7fed4fe10640 1 -- 172.21.15.40:0/155174023 wait complete. 2024-04-25T11:39:15.302 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T11:39:15.302 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:39:15.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.419+0000 7ffaf8b7d640 1 Processor -- start 2024-04-25T11:39:15.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.419+0000 7ffaf8b7d640 1 -- start start 2024-04-25T11:39:15.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.420+0000 7ffaf8b7d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffaf4154f40 0x7ffaf40c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.420+0000 7ffaf8b7d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffaf415ef60 0x7ffaf40cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.420+0000 7ffaf8b7d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf40c92e0 con 0x7ffaf40c97a0 2024-04-25T11:39:15.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.420+0000 7ffaf8b7d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ffaf406cfe0 con 0x7ffaf4154f40 2024-04-25T11:39:15.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.420+0000 7ffaf8b7d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ffaf406d2c0 con 0x7ffaf415ef60 2024-04-25T11:39:15.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.420+0000 7ffaf259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffaf4154f40 0x7ffaf40c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.420+0000 7ffaf259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffaf4154f40 0x7ffaf40c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37670/0 (socket says 172.21.15.40:37670) 2024-04-25T11:39:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.420+0000 7ffaf259c640 1 -- 172.21.15.40:0/1590840942 learned_addr learned my addr 172.21.15.40:0/1590840942 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.421+0000 7ffaf2d9d640 1 --2- 172.21.15.40:0/1590840942 >> v2:172.21.15.62:3300/0 conn(0x7ffaf415ef60 0x7ffaf40cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.421+0000 7ffaf259c640 1 -- 172.21.15.40:0/1590840942 >> v2:172.21.15.62:3300/0 conn(0x7ffaf415ef60 msgr2=0x7ffaf40cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.421+0000 7ffaf259c640 1 --2- 172.21.15.40:0/1590840942 >> v2:172.21.15.62:3300/0 conn(0x7ffaf415ef60 0x7ffaf40cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.421+0000 7ffaf259c640 1 -- 172.21.15.40:0/1590840942 >> v1:172.21.15.62:6789/0 conn(0x7ffaf40c97a0 legacy=0x7ffaf415ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.421+0000 7ffaf259c640 1 -- 172.21.15.40:0/1590840942 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf415e480 con 0x7ffaf4154f40 2024-04-25T11:39:15.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.421+0000 7ffaf259c640 1 --2- 172.21.15.40:0/1590840942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf4154f40 0x7ffaf40c8da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4071fc0 tx=0x7ffaec02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d227d441230621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf159a640 1 -- 172.21.15.40:0/1590840942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec037070 con 0x7ffaf4154f40 2024-04-25T11:39:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf159a640 1 -- 172.21.15.40:0/1590840942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaec02ae50 con 0x7ffaf4154f40 2024-04-25T11:39:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf159a640 1 -- 172.21.15.40:0/1590840942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec032860 con 0x7ffaf4154f40 2024-04-25T11:39:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/1590840942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf4154f40 msgr2=0x7ffaf40c8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf8b7d640 1 --2- 172.21.15.40:0/1590840942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf4154f40 0x7ffaf40c8da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4071fc0 tx=0x7ffaec02a300 comp rx=0 tx=0).stop 2024-04-25T11:39:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/1590840942 shutdown_connections 2024-04-25T11:39:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf8b7d640 1 --2- 172.21.15.40:0/1590840942 >> v2:172.21.15.62:3300/0 conn(0x7ffaf415ef60 0x7ffaf40cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf8b7d640 1 --2- 172.21.15.40:0/1590840942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf4154f40 0x7ffaf40c8da0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/1590840942 >> 172.21.15.40:0/1590840942 conn(0x7ffaf4070ae0 msgr2=0x7ffaf406fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/1590840942 shutdown_connections 2024-04-25T11:39:15.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.422+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/1590840942 wait complete. 2024-04-25T11:39:15.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.423+0000 7ffaf8b7d640 1 Processor -- start 2024-04-25T11:39:15.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf8b7d640 1 -- start start 2024-04-25T11:39:15.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf8b7d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf40c97a0 0x7ffaf415b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf8b7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf4154f40 0x7ffaf40c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf8b7d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf415ef60 0x7ffaf40ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf8b7d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4155320 con 0x7ffaf40c97a0 2024-04-25T11:39:15.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf8b7d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf415ea40 con 0x7ffaf4154f40 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf8b7d640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf406ca80 con 0x7ffaf415ef60 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf40c97a0 0x7ffaf415b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf40c97a0 0x7ffaf415b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37680/0 (socket says 172.21.15.40:37680) 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.424+0000 7ffaf259c640 1 -- 172.21.15.40:0/2068244882 learned_addr learned my addr 172.21.15.40:0/2068244882 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf1d9b640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf4154f40 0x7ffaf40c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf2d9d640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf415ef60 0x7ffaf40ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf259c640 1 -- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf415ef60 msgr2=0x7ffaf40ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf259c640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf415ef60 0x7ffaf40ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf259c640 1 -- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf4154f40 msgr2=0x7ffaf40c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf259c640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf4154f40 0x7ffaf40c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf259c640 1 -- 172.21.15.40:0/2068244882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf415bbd0 con 0x7ffaf40c97a0 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf1d9b640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf4154f40 0x7ffaf40c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf259c640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf40c97a0 0x7ffaf415b690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffaec02a340 tx=0x7ffaec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaeaffd640 1 -- 172.21.15.40:0/2068244882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec041070 con 0x7ffaf40c97a0 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaeaffd640 1 -- 172.21.15.40:0/2068244882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaec03c670 con 0x7ffaf40c97a0 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaeaffd640 1 -- 172.21.15.40:0/2068244882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec037040 con 0x7ffaf40c97a0 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf414fc20 con 0x7ffaf40c97a0 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.425+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf41f49b0 con 0x7ffaf40c97a0 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.426+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffaf4069200 con 0x7ffaf40c97a0 2024-04-25T11:39:15.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.426+0000 7ffaeaffd640 1 -- 172.21.15.40:0/2068244882 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffaec044070 con 0x7ffaf40c97a0 2024-04-25T11:39:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.427+0000 7ffaeaffd640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ffad406c8f0 0x7ffad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.427+0000 7ffaeaffd640 1 -- 172.21.15.40:0/2068244882 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ffaec0ac460 con 0x7ffaf40c97a0 2024-04-25T11:39:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.427+0000 7ffaeaffd640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7ffad4074c10 0x7ffad4077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.427+0000 7ffaf1d9b640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ffad406c8f0 0x7ffad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.427+0000 7ffaeaffd640 1 -- 172.21.15.40:0/2068244882 --> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffaec02cea0 con 0x7ffad4074c10 2024-04-25T11:39:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.427+0000 7ffaeaffd640 1 -- 172.21.15.40:0/2068244882 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ffaec0b3050 con 0x7ffaf40c97a0 2024-04-25T11:39:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.427+0000 7ffaf2d9d640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7ffad4074c10 0x7ffad4077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.428+0000 7ffaf1d9b640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ffad406c8f0 0x7ffad406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffaf414f470 tx=0x7ffadc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.428+0000 7ffaf2d9d640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7ffad4074c10 0x7ffad4077090 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.428+0000 7ffaeaffd640 1 -- 172.21.15.40:0/2068244882 <== osd.2 v2:172.21.15.40:6808/2321494678 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffaec02cea0 con 0x7ffad4074c10 2024-04-25T11:39:15.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.441+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 --> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffaf414e790 con 0x7ffad4074c10 2024-04-25T11:39:15.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaeaffd640 1 -- 172.21.15.40:0/2068244882 <== osd.2 v2:172.21.15.40:6808/2321494678 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffaf414e790 con 0x7ffad4074c10 2024-04-25T11:39:15.429 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:39:15.429 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:39:15.429 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:39:15.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7ffad4074c10 msgr2=0x7ffad4077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.40:6808/2321494678,v1:172.21.15.40:6809/2321494678] conn(0x7ffad4074c10 0x7ffad4077090 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ffad406c8f0 msgr2=0x7ffad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7ffad406c8f0 0x7ffad406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffaf414f470 tx=0x7ffadc009040 comp rx=0 tx=0).stop 2024-04-25T11:39:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf40c97a0 msgr2=0x7ffaf415b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 --2- 172.21.15.40:0/2068244882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf40c97a0 0x7ffaf415b690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffaec02a340 tx=0x7ffaec0046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf2d9d640 1 -- 172.21.15.40:0/2068244882 reap_dead start 2024-04-25T11:39:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 shutdown_connections 2024-04-25T11:39:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 >> 172.21.15.40:0/2068244882 conn(0x7ffaf4070ae0 msgr2=0x7ffaf4153790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 shutdown_connections 2024-04-25T11:39:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.442+0000 7ffaf8b7d640 1 -- 172.21.15.40:0/2068244882 wait complete. 2024-04-25T11:39:15.440 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T11:39:15.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.554+0000 7f1d1f660640 1 Processor -- start 2024-04-25T11:39:15.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.554+0000 7f1d1f660640 1 -- start start 2024-04-25T11:39:15.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.555+0000 7f1d1f660640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d180ca5c0 0x7f1d180cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.555+0000 7f1d1f660640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180d0530 0x7f1d1815e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.555+0000 7f1d1f660640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1d1815eaa0 0x7f1d180b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.555+0000 7f1d1f660640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1d1815bf60 con 0x7f1d180ca5c0 2024-04-25T11:39:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.555+0000 7f1d1f660640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f1d18155750 con 0x7f1d1815eaa0 2024-04-25T11:39:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.555+0000 7f1d1f660640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1d180ca9a0 con 0x7f1d180d0530 2024-04-25T11:39:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.555+0000 7f1d1d3fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d180ca5c0 0x7f1d180cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.555+0000 7f1d1d3fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1d180ca5c0 0x7f1d180cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37684/0 (socket says 172.21.15.40:37684) 2024-04-25T11:39:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.555+0000 7f1d1d3fc640 1 -- 172.21.15.40:0/1653722610 learned_addr learned my addr 172.21.15.40:0/1653722610 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d1dbfd640 1 --2- 172.21.15.40:0/1653722610 >> v2:172.21.15.62:3300/0 conn(0x7f1d1815eaa0 0x7f1d180b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d1cbfb640 1 --2- 172.21.15.40:0/1653722610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180d0530 0x7f1d1815e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d1d3fc640 1 -- 172.21.15.40:0/1653722610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180d0530 msgr2=0x7f1d1815e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d1d3fc640 1 --2- 172.21.15.40:0/1653722610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180d0530 0x7f1d1815e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d1d3fc640 1 -- 172.21.15.40:0/1653722610 >> v2:172.21.15.62:3300/0 conn(0x7f1d1815eaa0 msgr2=0x7f1d180b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d1d3fc640 1 --2- 172.21.15.40:0/1653722610 >> v2:172.21.15.62:3300/0 conn(0x7f1d1815eaa0 0x7f1d180b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d1d3fc640 1 -- 172.21.15.40:0/1653722610 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d180d0de0 con 0x7f1d180ca5c0 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d1dbfd640 1 --2- 172.21.15.40:0/1653722610 >> v2:172.21.15.62:3300/0 conn(0x7f1d1815eaa0 0x7f1d180b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d1d3fc640 1 --2- 172.21.15.40:0/1653722610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180ca5c0 0x7f1d180cb520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1d180d1470 tx=0x7f1d1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4832c184d7b40e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d0ffff640 1 -- 172.21.15.40:0/1653722610 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14037070 con 0x7f1d180ca5c0 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d0ffff640 1 -- 172.21.15.40:0/1653722610 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d14032440 con 0x7f1d180ca5c0 2024-04-25T11:39:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.556+0000 7f1d0ffff640 1 -- 172.21.15.40:0/1653722610 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14032810 con 0x7f1d180ca5c0 2024-04-25T11:39:15.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.557+0000 7f1d1f660640 1 -- 172.21.15.40:0/1653722610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180ca5c0 msgr2=0x7f1d180cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.557+0000 7f1d1f660640 1 --2- 172.21.15.40:0/1653722610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180ca5c0 0x7f1d180cb520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1d180d1470 tx=0x7f1d1402a270 comp rx=0 tx=0).stop 2024-04-25T11:39:15.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.557+0000 7f1d1f660640 1 -- 172.21.15.40:0/1653722610 shutdown_connections 2024-04-25T11:39:15.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.557+0000 7f1d1f660640 1 --2- 172.21.15.40:0/1653722610 >> v2:172.21.15.62:3300/0 conn(0x7f1d1815eaa0 0x7f1d180b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.557+0000 7f1d1f660640 1 --2- 172.21.15.40:0/1653722610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180d0530 0x7f1d1815e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.557+0000 7f1d1f660640 1 --2- 172.21.15.40:0/1653722610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180ca5c0 0x7f1d180cb520 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.557+0000 7f1d1f660640 1 -- 172.21.15.40:0/1653722610 >> 172.21.15.40:0/1653722610 conn(0x7f1d1806c5f0 msgr2=0x7f1d1806e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.557+0000 7f1d1f660640 1 -- 172.21.15.40:0/1653722610 shutdown_connections 2024-04-25T11:39:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.557+0000 7f1d1f660640 1 -- 172.21.15.40:0/1653722610 wait complete. 2024-04-25T11:39:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.558+0000 7f1d1f660640 1 Processor -- start 2024-04-25T11:39:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.558+0000 7f1d1f660640 1 -- start start 2024-04-25T11:39:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1f660640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180ca5c0 0x7f1d1815d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1f660640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180d0530 0x7f1d181545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1f660640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1815eaa0 0x7f1d180cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1f660640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1815b780 con 0x7f1d180d0530 2024-04-25T11:39:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1f660640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1815b4a0 con 0x7f1d1815eaa0 2024-04-25T11:39:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1f660640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1815ba60 con 0x7f1d180ca5c0 2024-04-25T11:39:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1cbfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180d0530 0x7f1d181545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1cbfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180d0530 0x7f1d181545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37688/0 (socket says 172.21.15.40:37688) 2024-04-25T11:39:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1cbfb640 1 -- 172.21.15.40:0/2611729774 learned_addr learned my addr 172.21.15.40:0/2611729774 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1dbfd640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1815eaa0 0x7f1d180cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1d3fc640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180ca5c0 0x7f1d1815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1cbfb640 1 -- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180ca5c0 msgr2=0x7f1d1815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1cbfb640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180ca5c0 0x7f1d1815d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1cbfb640 1 -- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1815eaa0 msgr2=0x7f1d180cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1cbfb640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1815eaa0 0x7f1d180cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.559+0000 7f1d1cbfb640 1 -- 172.21.15.40:0/2611729774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1815db80 con 0x7f1d180d0530 2024-04-25T11:39:15.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.560+0000 7f1d1cbfb640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180d0530 0x7f1d181545d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c5a0 tx=0x7f1d0800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.560+0000 7f1d0dffb640 1 -- 172.21.15.40:0/2611729774 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08015070 con 0x7f1d180d0530 2024-04-25T11:39:15.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.560+0000 7f1d0dffb640 1 -- 172.21.15.40:0/2611729774 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d08004220 con 0x7f1d180d0530 2024-04-25T11:39:15.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.560+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d181f1040 con 0x7f1d180d0530 2024-04-25T11:39:15.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.560+0000 7f1d0dffb640 1 -- 172.21.15.40:0/2611729774 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08003610 con 0x7f1d180d0530 2024-04-25T11:39:15.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.560+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1815d360 con 0x7f1d180d0530 2024-04-25T11:39:15.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.561+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1815d170 con 0x7f1d180d0530 2024-04-25T11:39:15.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.561+0000 7f1d0dffb640 1 -- 172.21.15.40:0/2611729774 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d080037b0 con 0x7f1d180d0530 2024-04-25T11:39:15.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.562+0000 7f1d0dffb640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1cfc06c920 0x7f1cfc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.562+0000 7f1d0dffb640 1 -- 172.21.15.40:0/2611729774 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1d0808b7c0 con 0x7f1d180d0530 2024-04-25T11:39:15.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.562+0000 7f1d1d3fc640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1cfc06c920 0x7f1cfc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.563+0000 7f1d1d3fc640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1cfc06c920 0x7f1cfc06ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1d14034000 tx=0x7f1d1402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.566+0000 7f1d0dffb640 1 -- 172.21.15.40:0/2611729774 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d08059520 con 0x7f1d180d0530 2024-04-25T11:39:15.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.752+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f1d1804fce0 con 0x7f1cfc06c920 2024-04-25T11:39:15.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.753+0000 7f1d0dffb640 1 -- 172.21.15.40:0/2611729774 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1d1804fce0 con 0x7f1cfc06c920 2024-04-25T11:39:15.740 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T11:39:15.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1cfc06c920 msgr2=0x7f1cfc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1cfc06c920 0x7f1cfc06ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1d14034000 tx=0x7f1d1402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:15.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180d0530 msgr2=0x7f1d181545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180d0530 0x7f1d181545d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c5a0 tx=0x7f1d0800ca60 comp rx=0 tx=0).stop 2024-04-25T11:39:15.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 shutdown_connections 2024-04-25T11:39:15.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1cfc06c920 0x7f1cfc06ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1815eaa0 0x7f1d180cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d180d0530 0x7f1d181545d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 --2- 172.21.15.40:0/2611729774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d180ca5c0 0x7f1d1815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 >> 172.21.15.40:0/2611729774 conn(0x7f1d1806c5f0 msgr2=0x7f1d180b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 shutdown_connections 2024-04-25T11:39:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.755+0000 7f1d1f660640 1 -- 172.21.15.40:0/2611729774 wait complete. 2024-04-25T11:39:15.753 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T11:39:15.753 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:39:15.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.871+0000 7efdc179b640 1 Processor -- start 2024-04-25T11:39:15.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.871+0000 7efdc179b640 1 -- start start 2024-04-25T11:39:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdc179b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdbc155130 0x7efdbc15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdc179b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdbc0cb280 0x7efdbc0c5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdc179b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdbc06d450 con 0x7efdbc0c5a70 2024-04-25T11:39:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdc179b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efdbc15f170 con 0x7efdbc0cb280 2024-04-25T11:39:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdc179b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc0cca30 con 0x7efdbc155130 2024-04-25T11:39:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdba7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efdbc0cb280 0x7efdbc0c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdbb7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efdbc0c5a70 0x7efdbc0ccf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34544/0 (socket says 172.21.15.40:34544) 2024-04-25T11:39:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdbb7fe640 1 -- 172.21.15.40:0/476565956 learned_addr learned my addr 172.21.15.40:0/476565956 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdbaffd640 1 --2- 172.21.15.40:0/476565956 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdbc155130 0x7efdbc15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdba7fc640 1 -- 172.21.15.40:0/476565956 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdbc155130 msgr2=0x7efdbc15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdba7fc640 1 --2- 172.21.15.40:0/476565956 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdbc155130 0x7efdbc15ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdba7fc640 1 -- 172.21.15.40:0/476565956 >> v1:172.21.15.40:6789/0 conn(0x7efdbc0c5a70 legacy=0x7efdbc0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdba7fc640 1 -- 172.21.15.40:0/476565956 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc06c690 con 0x7efdbc0cb280 2024-04-25T11:39:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.872+0000 7efdb9ffb640 1 -- 172.21.15.40:0/476565956 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4102816579 0 0) 0x7efdbc06d450 con 0x7efdbc0c5a70 2024-04-25T11:39:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.873+0000 7efdba7fc640 1 --2- 172.21.15.40:0/476565956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc0cb280 0x7efdbc0c5530 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efda40097a0 tx=0x7efda402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb631e4be32a735f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.873+0000 7efdb9ffb640 1 -- 172.21.15.40:0/476565956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4037070 con 0x7efdbc0cb280 2024-04-25T11:39:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.873+0000 7efdb9ffb640 1 -- 172.21.15.40:0/476565956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda4032440 con 0x7efdbc0cb280 2024-04-25T11:39:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.873+0000 7efdb9ffb640 1 -- 172.21.15.40:0/476565956 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4032810 con 0x7efdbc0cb280 2024-04-25T11:39:15.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.874+0000 7efdc179b640 1 -- 172.21.15.40:0/476565956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc0cb280 msgr2=0x7efdbc0c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.874+0000 7efdc179b640 1 --2- 172.21.15.40:0/476565956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc0cb280 0x7efdbc0c5530 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efda40097a0 tx=0x7efda402a180 comp rx=0 tx=0).stop 2024-04-25T11:39:15.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.874+0000 7efdc179b640 1 -- 172.21.15.40:0/476565956 shutdown_connections 2024-04-25T11:39:15.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.874+0000 7efdc179b640 1 --2- 172.21.15.40:0/476565956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc0cb280 0x7efdbc0c5530 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.874+0000 7efdc179b640 1 --2- 172.21.15.40:0/476565956 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdbc155130 0x7efdbc15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.874+0000 7efdc179b640 1 -- 172.21.15.40:0/476565956 >> 172.21.15.40:0/476565956 conn(0x7efdbc0676b0 msgr2=0x7efdbc067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.874+0000 7efdc179b640 1 -- 172.21.15.40:0/476565956 shutdown_connections 2024-04-25T11:39:15.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.874+0000 7efdc179b640 1 -- 172.21.15.40:0/476565956 wait complete. 2024-04-25T11:39:15.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.875+0000 7efdc179b640 1 Processor -- start 2024-04-25T11:39:15.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.875+0000 7efdc179b640 1 -- start start 2024-04-25T11:39:15.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdc179b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdbc0c5a70 0x7efdbc14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdc179b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdbc0cb280 0x7efdbc1f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdc179b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc155130 0x7efdbc145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdc179b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc06cf30 con 0x7efdbc155130 2024-04-25T11:39:15.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdc179b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc06cc50 con 0x7efdbc0c5a70 2024-04-25T11:39:15.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdc179b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc0cd620 con 0x7efdbc0cb280 2024-04-25T11:39:15.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc155130 0x7efdbc145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc155130 0x7efdbc145010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37700/0 (socket says 172.21.15.40:37700) 2024-04-25T11:39:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbb7fe640 1 -- 172.21.15.40:0/1949259835 learned_addr learned my addr 172.21.15.40:0/1949259835 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbaffd640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdbc0c5a70 0x7efdbc14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdba7fc640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdbc0cb280 0x7efdbc1f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbb7fe640 1 -- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdbc0cb280 msgr2=0x7efdbc1f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbb7fe640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efdbc0cb280 0x7efdbc1f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbb7fe640 1 -- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdbc0c5a70 msgr2=0x7efdbc14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbb7fe640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdbc0c5a70 0x7efdbc14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbb7fe640 1 -- 172.21.15.40:0/1949259835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc1472f0 con 0x7efdbc155130 2024-04-25T11:39:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdbb7fe640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc155130 0x7efdbc145010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efdac00c760 tx=0x7efdac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdb37fe640 1 -- 172.21.15.40:0/1949259835 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac018070 con 0x7efdbc155130 2024-04-25T11:39:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.876+0000 7efdb37fe640 1 -- 172.21.15.40:0/1949259835 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac00a640 con 0x7efdbc155130 2024-04-25T11:39:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.877+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdbc1f4c20 con 0x7efdbc155130 2024-04-25T11:39:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.877+0000 7efdb37fe640 1 -- 172.21.15.40:0/1949259835 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac00aa30 con 0x7efdbc155130 2024-04-25T11:39:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.877+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdbc15dc00 con 0x7efdbc155130 2024-04-25T11:39:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.877+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdbc0cab80 con 0x7efdbc155130 2024-04-25T11:39:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.878+0000 7efdb37fe640 1 -- 172.21.15.40:0/1949259835 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdac01b070 con 0x7efdbc155130 2024-04-25T11:39:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.878+0000 7efdb37fe640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7efd9c06c920 0x7efd9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.879+0000 7efdb37fe640 1 -- 172.21.15.40:0/1949259835 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7efdac08b140 con 0x7efdbc155130 2024-04-25T11:39:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.879+0000 7efdbaffd640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7efd9c06c920 0x7efd9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.879+0000 7efdb37fe640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7efd9c074c40 0x7efd9c0770c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.879+0000 7efdb37fe640 1 -- 172.21.15.40:0/1949259835 --> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd9c077790 con 0x7efd9c074c40 2024-04-25T11:39:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.879+0000 7efdb37fe640 1 -- 172.21.15.40:0/1949259835 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efdac092050 con 0x7efdbc155130 2024-04-25T11:39:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.879+0000 7efdba7fc640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7efd9c074c40 0x7efd9c0770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.879+0000 7efdbaffd640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7efd9c06c920 0x7efd9c06ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009aa0 tx=0x7efdb4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.879+0000 7efdba7fc640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7efd9c074c40 0x7efd9c0770c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.880+0000 7efdb37fe640 1 -- 172.21.15.40:0/1949259835 <== osd.3 v2:172.21.15.40:6818/364049698 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd9c077790 con 0x7efd9c074c40 2024-04-25T11:39:15.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.893+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 --> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efdbc068bb0 con 0x7efd9c074c40 2024-04-25T11:39:15.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.893+0000 7efdb37fe640 1 -- 172.21.15.40:0/1949259835 <== osd.3 v2:172.21.15.40:6818/364049698 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efdbc068bb0 con 0x7efd9c074c40 2024-04-25T11:39:15.880 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:39:15.881 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:39:15.881 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:39:15.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7efd9c074c40 msgr2=0x7efd9c0770c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.40:6818/364049698,v1:172.21.15.40:6820/364049698] conn(0x7efd9c074c40 0x7efd9c0770c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7efd9c06c920 msgr2=0x7efd9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7efd9c06c920 0x7efd9c06ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009aa0 tx=0x7efdb4009190 comp rx=0 tx=0).stop 2024-04-25T11:39:15.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc155130 msgr2=0x7efdbc145010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 --2- 172.21.15.40:0/1949259835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efdbc155130 0x7efdbc145010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efdac00c760 tx=0x7efdac00cc20 comp rx=0 tx=0).stop 2024-04-25T11:39:15.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdbb7fe640 1 -- 172.21.15.40:0/1949259835 reap_dead start 2024-04-25T11:39:15.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 shutdown_connections 2024-04-25T11:39:15.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 >> 172.21.15.40:0/1949259835 conn(0x7efdbc0676b0 msgr2=0x7efdbc067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 shutdown_connections 2024-04-25T11:39:15.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:15.894+0000 7efdc179b640 1 -- 172.21.15.40:0/1949259835 wait complete. 2024-04-25T11:39:15.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T11:39:15.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.006+0000 7f91d8e59640 1 Processor -- start 2024-04-25T11:39:15.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.006+0000 7f91d8e59640 1 -- start start 2024-04-25T11:39:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d8e59640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 0x7f91d40cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d8e59640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91d40ccd00 0x7f91d40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d8e59640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91d40cd6a0 con 0x7f91d40cb240 2024-04-25T11:39:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d8e59640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f91d415ae60 con 0x7f91d40ccd00 2024-04-25T11:39:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d8e59640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f91d415b140 con 0x7f91d40c7b70 2024-04-25T11:39:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d2d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91d40ccd00 0x7f91d40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d2d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91d40ccd00 0x7f91d40cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37712/0 (socket says 172.21.15.40:37712) 2024-04-25T11:39:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d2d9d640 1 -- 172.21.15.40:0/3246586875 learned_addr learned my addr 172.21.15.40:0/3246586875 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d1d9b640 1 --2- 172.21.15.40:0/3246586875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 0x7f91d40cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d2d9d640 1 -- 172.21.15.40:0/3246586875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 msgr2=0x7f91d40cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d2d9d640 1 --2- 172.21.15.40:0/3246586875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 0x7f91d40cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d2d9d640 1 -- 172.21.15.40:0/3246586875 >> v1:172.21.15.62:6789/0 conn(0x7f91d40cb240 legacy=0x7f91d40cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d2d9d640 1 -- 172.21.15.40:0/3246586875 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d415b420 con 0x7f91d40ccd00 2024-04-25T11:39:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.007+0000 7f91d1d9b640 1 --2- 172.21.15.40:0/3246586875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 0x7f91d40cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.008+0000 7f91d2d9d640 1 --2- 172.21.15.40:0/3246586875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40ccd00 0x7f91d40cd160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009870 tx=0x7f91cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b359a4d4b7c51f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.008+0000 7f91d159a640 1 -- 172.21.15.40:0/3246586875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc037070 con 0x7f91d40ccd00 2024-04-25T11:39:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.008+0000 7f91d159a640 1 -- 172.21.15.40:0/3246586875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91cc032440 con 0x7f91d40ccd00 2024-04-25T11:39:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.008+0000 7f91d159a640 1 -- 172.21.15.40:0/3246586875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc032810 con 0x7f91d40ccd00 2024-04-25T11:39:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.009+0000 7f91d8e59640 1 -- 172.21.15.40:0/3246586875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40ccd00 msgr2=0x7f91d40cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.009+0000 7f91d8e59640 1 --2- 172.21.15.40:0/3246586875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40ccd00 0x7f91d40cd160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009870 tx=0x7f91cc02a270 comp rx=0 tx=0).stop 2024-04-25T11:39:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.009+0000 7f91d8e59640 1 -- 172.21.15.40:0/3246586875 shutdown_connections 2024-04-25T11:39:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.009+0000 7f91d8e59640 1 --2- 172.21.15.40:0/3246586875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40ccd00 0x7f91d40cd160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.009+0000 7f91d8e59640 1 --2- 172.21.15.40:0/3246586875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 0x7f91d40cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.009+0000 7f91d8e59640 1 -- 172.21.15.40:0/3246586875 >> 172.21.15.40:0/3246586875 conn(0x7f91d406c3d0 msgr2=0x7f91d406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.009+0000 7f91d8e59640 1 -- 172.21.15.40:0/3246586875 shutdown_connections 2024-04-25T11:39:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.009+0000 7f91d8e59640 1 -- 172.21.15.40:0/3246586875 wait complete. 2024-04-25T11:39:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.010+0000 7f91d8e59640 1 Processor -- start 2024-04-25T11:39:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.010+0000 7f91d8e59640 1 -- start start 2024-04-25T11:39:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d8e59640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 0x7f91d4153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d8e59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40cb240 0x7f91d41542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d8e59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d40ccd00 0x7f91d41f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d8e59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91d41552b0 con 0x7f91d40cb240 2024-04-25T11:39:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d8e59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91d415b9e0 con 0x7f91d40ccd00 2024-04-25T11:39:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d8e59640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f91d41f17b0 con 0x7f91d40c7b70 2024-04-25T11:39:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d1d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40cb240 0x7f91d41542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d1d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40cb240 0x7f91d41542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37716/0 (socket says 172.21.15.40:37716) 2024-04-25T11:39:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d1d9b640 1 -- 172.21.15.40:0/903766339 learned_addr learned my addr 172.21.15.40:0/903766339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d259c640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 0x7f91d4153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.011+0000 7f91d2d9d640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d40ccd00 0x7f91d41f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91d1d9b640 1 -- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 msgr2=0x7f91d4153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91d1d9b640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 0x7f91d4153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91d1d9b640 1 -- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d40ccd00 msgr2=0x7f91d41f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91d1d9b640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d40ccd00 0x7f91d41f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91d1d9b640 1 -- 172.21.15.40:0/903766339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d4154800 con 0x7f91d40cb240 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91d1d9b640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40cb240 0x7f91d41542c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00e810 tx=0x7f91bc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91caffd640 1 -- 172.21.15.40:0/903766339 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00f040 con 0x7f91d40cb240 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91caffd640 1 -- 172.21.15.40:0/903766339 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc004230 con 0x7f91d40cb240 2024-04-25T11:39:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d4159790 con 0x7f91d40cb240 2024-04-25T11:39:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91caffd640 1 -- 172.21.15.40:0/903766339 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc005990 con 0x7f91d40cb240 2024-04-25T11:39:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.012+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d4159d50 con 0x7f91d40cb240 2024-04-25T11:39:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.013+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d40cacb0 con 0x7f91d40cb240 2024-04-25T11:39:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.013+0000 7f91caffd640 1 -- 172.21.15.40:0/903766339 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91bc005b30 con 0x7f91d40cb240 2024-04-25T11:39:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.014+0000 7f91caffd640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91b406c8f0 0x7f91b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.014+0000 7f91caffd640 1 -- 172.21.15.40:0/903766339 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f91bc08bb10 con 0x7f91d40cb240 2024-04-25T11:39:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.014+0000 7f91d259c640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91b406c8f0 0x7f91b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.014+0000 7f91d259c640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91b406c8f0 0x7f91b406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91c4001f10 tx=0x7f91c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.018+0000 7f91caffd640 1 -- 172.21.15.40:0/903766339 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91bc020020 con 0x7f91d40cb240 2024-04-25T11:39:16.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.202+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f91d404fd50 con 0x7f91b406c8f0 2024-04-25T11:39:16.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.203+0000 7f91caffd640 1 -- 172.21.15.40:0/903766339 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f91d404fd50 con 0x7f91b406c8f0 2024-04-25T11:39:16.190 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T11:39:16.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91b406c8f0 msgr2=0x7f91b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91b406c8f0 0x7f91b406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91c4001f10 tx=0x7f91c4009040 comp rx=0 tx=0).stop 2024-04-25T11:39:16.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40cb240 msgr2=0x7f91d41542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40cb240 0x7f91d41542c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00e810 tx=0x7f91bc00ecd0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 shutdown_connections 2024-04-25T11:39:16.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f91b406c8f0 0x7f91b406edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d40ccd00 0x7f91d41f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91d40cb240 0x7f91d41542c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 --2- 172.21.15.40:0/903766339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91d40c7b70 0x7f91d4153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 >> 172.21.15.40:0/903766339 conn(0x7f91d406c3d0 msgr2=0x7f91d4074b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 shutdown_connections 2024-04-25T11:39:16.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.205+0000 7f91d8e59640 1 -- 172.21.15.40:0/903766339 wait complete. 2024-04-25T11:39:16.203 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T11:39:16.203 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:39:16.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.321+0000 7f2509f0f640 1 Processor -- start 2024-04-25T11:39:16.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.321+0000 7f2509f0f640 1 -- start start 2024-04-25T11:39:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f2509f0f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2504154f20 0x7f250415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f2509f0f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f25040cd630 0x7f25040cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f2509f0f640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25040cbc80 con 0x7f2504155300 2024-04-25T11:39:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f2509f0f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f25040ccbe0 con 0x7f2504154f20 2024-04-25T11:39:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f2509f0f640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f25040cd1a0 con 0x7f25040cd630 2024-04-25T11:39:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f25037fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2504154f20 0x7f250415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f25037fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2504154f20 0x7f250415ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37718/0 (socket says 172.21.15.40:37718) 2024-04-25T11:39:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f25037fe640 1 -- 172.21.15.40:0/283483304 learned_addr learned my addr 172.21.15.40:0/283483304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f2503fff640 1 --2- 172.21.15.40:0/283483304 >> v2:172.21.15.62:3300/0 conn(0x7f25040cd630 0x7f25040cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f25037fe640 1 -- 172.21.15.40:0/283483304 >> v2:172.21.15.62:3300/0 conn(0x7f25040cd630 msgr2=0x7f25040cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f25037fe640 1 --2- 172.21.15.40:0/283483304 >> v2:172.21.15.62:3300/0 conn(0x7f25040cd630 0x7f25040cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f25037fe640 1 -- 172.21.15.40:0/283483304 >> v1:172.21.15.62:6789/0 conn(0x7f2504155300 legacy=0x7f25040cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.322+0000 7f25037fe640 1 -- 172.21.15.40:0/283483304 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250415b5e0 con 0x7f2504154f20 2024-04-25T11:39:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.323+0000 7f25037fe640 1 --2- 172.21.15.40:0/283483304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 0x7f250415ada0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f250406cf30 tx=0x7f24f402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eae62117a98a4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.323+0000 7f25027fc640 1 -- 172.21.15.40:0/283483304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4036070 con 0x7f2504154f20 2024-04-25T11:39:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.323+0000 7f25027fc640 1 -- 172.21.15.40:0/283483304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4031600 con 0x7f2504154f20 2024-04-25T11:39:16.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.323+0000 7f25027fc640 1 -- 172.21.15.40:0/283483304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f40319f0 con 0x7f2504154f20 2024-04-25T11:39:16.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.324+0000 7f2509f0f640 1 -- 172.21.15.40:0/283483304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 msgr2=0x7f250415ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.324+0000 7f2509f0f640 1 --2- 172.21.15.40:0/283483304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 0x7f250415ada0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f250406cf30 tx=0x7f24f402a7c0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.324+0000 7f2509f0f640 1 -- 172.21.15.40:0/283483304 shutdown_connections 2024-04-25T11:39:16.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.324+0000 7f2509f0f640 1 --2- 172.21.15.40:0/283483304 >> v2:172.21.15.62:3300/0 conn(0x7f25040cd630 0x7f25040cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.324+0000 7f2509f0f640 1 --2- 172.21.15.40:0/283483304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 0x7f250415ada0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.324+0000 7f2509f0f640 1 -- 172.21.15.40:0/283483304 >> 172.21.15.40:0/283483304 conn(0x7f250406c9a0 msgr2=0x7f250406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.324+0000 7f2509f0f640 1 -- 172.21.15.40:0/283483304 shutdown_connections 2024-04-25T11:39:16.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.324+0000 7f2509f0f640 1 -- 172.21.15.40:0/283483304 wait complete. 2024-04-25T11:39:16.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.325+0000 7f2509f0f640 1 Processor -- start 2024-04-25T11:39:16.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.325+0000 7f2509f0f640 1 -- start start 2024-04-25T11:39:16.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2509f0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25040cd630 0x7f25041f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2509f0f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 0x7f25040cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2509f0f640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2504155300 0x7f25040c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2509f0f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25040cce30 con 0x7f2504154f20 2024-04-25T11:39:16.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2509f0f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f250415bba0 con 0x7f25040cd630 2024-04-25T11:39:16.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2509f0f640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f250415b2e0 con 0x7f2504155300 2024-04-25T11:39:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2502ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 0x7f25040cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2502ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 0x7f25040cef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37724/0 (socket says 172.21.15.40:37724) 2024-04-25T11:39:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2502ffd640 1 -- 172.21.15.40:0/1599063384 learned_addr learned my addr 172.21.15.40:0/1599063384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2503fff640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2504155300 0x7f25040c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f25037fe640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25040cd630 0x7f25041f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2502ffd640 1 -- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2504155300 msgr2=0x7f25040c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2502ffd640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2504155300 0x7f25040c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2502ffd640 1 -- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25040cd630 msgr2=0x7f25041f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2502ffd640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25040cd630 0x7f25041f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f2502ffd640 1 -- 172.21.15.40:0/1599063384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25041521c0 con 0x7f2504154f20 2024-04-25T11:39:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.326+0000 7f25037fe640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25040cd630 0x7f25041f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.327+0000 7f2502ffd640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 0x7f25040cef20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f24f800c3f0 tx=0x7f24f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.327+0000 7f24ebfff640 1 -- 172.21.15.40:0/1599063384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80052d0 con 0x7f2504154f20 2024-04-25T11:39:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.327+0000 7f24ebfff640 1 -- 172.21.15.40:0/1599063384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f8005490 con 0x7f2504154f20 2024-04-25T11:39:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.327+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250414bde0 con 0x7f2504154f20 2024-04-25T11:39:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.327+0000 7f24ebfff640 1 -- 172.21.15.40:0/1599063384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8007260 con 0x7f2504154f20 2024-04-25T11:39:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.327+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25040ce440 con 0x7f2504154f20 2024-04-25T11:39:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.328+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2504068bb0 con 0x7f2504154f20 2024-04-25T11:39:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.328+0000 7f24ebfff640 1 -- 172.21.15.40:0/1599063384 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24f8007400 con 0x7f2504154f20 2024-04-25T11:39:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.329+0000 7f24ebfff640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f24f006c8f0 0x7f24f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.329+0000 7f24ebfff640 1 -- 172.21.15.40:0/1599063384 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f24f808afe0 con 0x7f2504154f20 2024-04-25T11:39:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.329+0000 7f25037fe640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f24f006c8f0 0x7f24f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.329+0000 7f24ebfff640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f24f0074c70 0x7f24f00770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.329+0000 7f24ebfff640 1 -- 172.21.15.40:0/1599063384 --> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24f8009e50 con 0x7f24f0074c70 2024-04-25T11:39:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.329+0000 7f24ebfff640 1 -- 172.21.15.40:0/1599063384 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f24f8092050 con 0x7f2504154f20 2024-04-25T11:39:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.329+0000 7f2503fff640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f24f0074c70 0x7f24f00770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.329+0000 7f25037fe640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f24f006c8f0 0x7f24f006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24f4033000 tx=0x7f24f4009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.329+0000 7f2503fff640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f24f0074c70 0x7f24f00770f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.330+0000 7f24ebfff640 1 -- 172.21.15.40:0/1599063384 <== osd.4 v2:172.21.15.62:6800/2239863089 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24f8009e50 con 0x7f24f0074c70 2024-04-25T11:39:16.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.343+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 --> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f25040cbe80 con 0x7f24f0074c70 2024-04-25T11:39:16.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f24ebfff640 1 -- 172.21.15.40:0/1599063384 <== osd.4 v2:172.21.15.62:6800/2239863089 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f25040cbe80 con 0x7f24f0074c70 2024-04-25T11:39:16.331 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:39:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:39:16.331 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:39:16.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f24f0074c70 msgr2=0x7f24f00770f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6800/2239863089,v1:172.21.15.62:6801/2239863089] conn(0x7f24f0074c70 0x7f24f00770f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f24f006c8f0 msgr2=0x7f24f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f24f006c8f0 0x7f24f006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24f4033000 tx=0x7f24f4009000 comp rx=0 tx=0).stop 2024-04-25T11:39:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 msgr2=0x7f25040cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 --2- 172.21.15.40:0/1599063384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2504154f20 0x7f25040cef20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f24f800c3f0 tx=0x7f24f800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2503fff640 1 -- 172.21.15.40:0/1599063384 reap_dead start 2024-04-25T11:39:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 shutdown_connections 2024-04-25T11:39:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 >> 172.21.15.40:0/1599063384 conn(0x7f250406c9a0 msgr2=0x7f250414f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 shutdown_connections 2024-04-25T11:39:16.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.344+0000 7f2509f0f640 1 -- 172.21.15.40:0/1599063384 wait complete. 2024-04-25T11:39:16.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T11:39:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.456+0000 7f970fb20640 1 Processor -- start 2024-04-25T11:39:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.456+0000 7f970fb20640 1 -- start start 2024-04-25T11:39:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970fb20640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 0x7f97080cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970fb20640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f970815ed40 0x7f970815b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970fb20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97080c7840 0x7f97080d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970fb20640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f970815bc90 con 0x7f97080c7840 2024-04-25T11:39:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970fb20640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f97080cb460 con 0x7f970815ed40 2024-04-25T11:39:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970fb20640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f97080cb740 con 0x7f9708155080 2024-04-25T11:39:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970e0bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97080c7840 0x7f97080d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970e0bd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97080c7840 0x7f97080d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37732/0 (socket says 172.21.15.40:37732) 2024-04-25T11:39:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970d8bc640 1 --2- 172.21.15.40:0/2986626417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 0x7f97080cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970e0bd640 1 -- 172.21.15.40:0/2986626417 learned_addr learned my addr 172.21.15.40:0/2986626417 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970d0bb640 1 --2- 172.21.15.40:0/2986626417 >> v2:172.21.15.62:3300/0 conn(0x7f970815ed40 0x7f970815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970d8bc640 1 -- 172.21.15.40:0/2986626417 >> v2:172.21.15.62:3300/0 conn(0x7f970815ed40 msgr2=0x7f970815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970d8bc640 1 --2- 172.21.15.40:0/2986626417 >> v2:172.21.15.62:3300/0 conn(0x7f970815ed40 0x7f970815b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970d8bc640 1 -- 172.21.15.40:0/2986626417 >> v2:172.21.15.40:3300/0 conn(0x7f97080c7840 msgr2=0x7f97080d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970d8bc640 1 --2- 172.21.15.40:0/2986626417 >> v2:172.21.15.40:3300/0 conn(0x7f97080c7840 0x7f97080d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970d8bc640 1 -- 172.21.15.40:0/2986626417 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970815af50 con 0x7f9708155080 2024-04-25T11:39:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.457+0000 7f970e0bd640 1 --2- 172.21.15.40:0/2986626417 >> v2:172.21.15.40:3300/0 conn(0x7f97080c7840 0x7f97080d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:39:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.458+0000 7f970d8bc640 1 --2- 172.21.15.40:0/2986626417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 0x7f97080cc720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f970806cc90 tx=0x7f970402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a73e2b1b2815cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.458+0000 7f970c8ba640 1 -- 172.21.15.40:0/2986626417 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704037070 con 0x7f9708155080 2024-04-25T11:39:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.458+0000 7f970c8ba640 1 -- 172.21.15.40:0/2986626417 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970402add0 con 0x7f9708155080 2024-04-25T11:39:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.458+0000 7f970c8ba640 1 -- 172.21.15.40:0/2986626417 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704032660 con 0x7f9708155080 2024-04-25T11:39:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.459+0000 7f970fb20640 1 -- 172.21.15.40:0/2986626417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 msgr2=0x7f97080cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.459+0000 7f970fb20640 1 --2- 172.21.15.40:0/2986626417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 0x7f97080cc720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f970806cc90 tx=0x7f970402a080 comp rx=0 tx=0).stop 2024-04-25T11:39:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.459+0000 7f970fb20640 1 -- 172.21.15.40:0/2986626417 shutdown_connections 2024-04-25T11:39:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.459+0000 7f970fb20640 1 --2- 172.21.15.40:0/2986626417 >> v2:172.21.15.40:3300/0 conn(0x7f97080c7840 0x7f97080d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.459+0000 7f970fb20640 1 --2- 172.21.15.40:0/2986626417 >> v2:172.21.15.62:3300/0 conn(0x7f970815ed40 0x7f970815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.459+0000 7f970fb20640 1 --2- 172.21.15.40:0/2986626417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 0x7f97080cc720 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.459+0000 7f970fb20640 1 -- 172.21.15.40:0/2986626417 >> 172.21.15.40:0/2986626417 conn(0x7f970806c680 msgr2=0x7f970806e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.459+0000 7f970fb20640 1 -- 172.21.15.40:0/2986626417 shutdown_connections 2024-04-25T11:39:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.459+0000 7f970fb20640 1 -- 172.21.15.40:0/2986626417 wait complete. 2024-04-25T11:39:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.460+0000 7f970fb20640 1 Processor -- start 2024-04-25T11:39:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.460+0000 7f970fb20640 1 -- start start 2024-04-25T11:39:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970fb20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97080c7840 0x7f97081f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970fb20640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 0x7f9708158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970fb20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970815ed40 0x7f9708152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970fb20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f970815f210 con 0x7f970815ed40 2024-04-25T11:39:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970fb20640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9708155460 con 0x7f97080c7840 2024-04-25T11:39:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970fb20640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f97080cc140 con 0x7f9708155080 2024-04-25T11:39:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970e0bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970815ed40 0x7f9708152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970d8bc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97080c7840 0x7f97081f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970e0bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970815ed40 0x7f9708152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37748/0 (socket says 172.21.15.40:37748) 2024-04-25T11:39:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970e0bd640 1 -- 172.21.15.40:0/3301604586 learned_addr learned my addr 172.21.15.40:0/3301604586 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970d0bb640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 0x7f9708158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970e0bd640 1 -- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 msgr2=0x7f9708158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970e0bd640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 0x7f9708158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970e0bd640 1 -- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97080c7840 msgr2=0x7f97081f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970e0bd640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97080c7840 0x7f97081f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970e0bd640 1 -- 172.21.15.40:0/3301604586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97080cfb20 con 0x7f970815ed40 2024-04-25T11:39:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.461+0000 7f970e0bd640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970815ed40 0x7f9708152520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f970000ef00 tx=0x7f970000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.462+0000 7f96fe7fc640 1 -- 172.21.15.40:0/3301604586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970000bb10 con 0x7f970815ed40 2024-04-25T11:39:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.462+0000 7f96fe7fc640 1 -- 172.21.15.40:0/3301604586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970000bcb0 con 0x7f970815ed40 2024-04-25T11:39:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.462+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97080d0070 con 0x7f970815ed40 2024-04-25T11:39:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.462+0000 7f96fe7fc640 1 -- 172.21.15.40:0/3301604586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970000be60 con 0x7f970815ed40 2024-04-25T11:39:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.462+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97080cfd90 con 0x7f970815ed40 2024-04-25T11:39:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.463+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970804fd50 con 0x7f970815ed40 2024-04-25T11:39:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.463+0000 7f96fe7fc640 1 -- 172.21.15.40:0/3301604586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f970001b030 con 0x7f970815ed40 2024-04-25T11:39:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.463+0000 7f96fe7fc640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f96ec06c8f0 0x7f96ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.464+0000 7f96fe7fc640 1 -- 172.21.15.40:0/3301604586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f970008ba90 con 0x7f970815ed40 2024-04-25T11:39:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.464+0000 7f970d8bc640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f96ec06c8f0 0x7f96ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.464+0000 7f970d8bc640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f96ec06c8f0 0x7f96ec06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9704034000 tx=0x7f970402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.468+0000 7f96fe7fc640 1 -- 172.21.15.40:0/3301604586 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9700022020 con 0x7f970815ed40 2024-04-25T11:39:16.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.651+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f97081e7e00 con 0x7f96ec06c8f0 2024-04-25T11:39:16.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.651+0000 7f96fe7fc640 1 -- 172.21.15.40:0/3301604586 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f970402acd0 con 0x7f96ec06c8f0 2024-04-25T11:39:16.639 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T11:39:16.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f96ec06c8f0 msgr2=0x7f96ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f96ec06c8f0 0x7f96ec06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9704034000 tx=0x7f970402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970815ed40 msgr2=0x7f9708152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970815ed40 0x7f9708152520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f970000ef00 tx=0x7f970000b480 comp rx=0 tx=0).stop 2024-04-25T11:39:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 shutdown_connections 2024-04-25T11:39:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f96ec06c8f0 0x7f96ec06edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970815ed40 0x7f9708152520 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9708155080 0x7f9708158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 --2- 172.21.15.40:0/3301604586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97080c7840 0x7f97081f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 >> 172.21.15.40:0/3301604586 conn(0x7f970806c680 msgr2=0x7f970806e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 shutdown_connections 2024-04-25T11:39:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.653+0000 7f970fb20640 1 -- 172.21.15.40:0/3301604586 wait complete. 2024-04-25T11:39:16.651 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T11:39:16.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:39:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.769+0000 7f992a491640 1 Processor -- start 2024-04-25T11:39:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.769+0000 7f992a491640 1 -- start start 2024-04-25T11:39:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f992a491640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 0x7f992415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f992a491640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9924155300 0x7f99240cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f992a491640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f992415b5e0 con 0x7f99240cd630 2024-04-25T11:39:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f992a491640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f99240ccbe0 con 0x7f9924155300 2024-04-25T11:39:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f992a491640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f99240cd1a0 con 0x7f9924154f20 2024-04-25T11:39:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f9923fff640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 0x7f992415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f99237fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9924155300 0x7f99240cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f9928a2e640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f99240cd630 0x7f99240cc0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:38572/0 (socket says 172.21.15.40:38572) 2024-04-25T11:39:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f9923fff640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 0x7f992415ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:36802/0 (socket says 172.21.15.40:36802) 2024-04-25T11:39:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f9928a2e640 1 -- 172.21.15.40:0/1794191082 learned_addr learned my addr 172.21.15.40:0/1794191082 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f9923fff640 1 -- 172.21.15.40:0/1794191082 >> v2:172.21.15.62:3300/0 conn(0x7f9924155300 msgr2=0x7f99240cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f9923fff640 1 --2- 172.21.15.40:0/1794191082 >> v2:172.21.15.62:3300/0 conn(0x7f9924155300 0x7f99240cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.770+0000 7f9923fff640 1 -- 172.21.15.40:0/1794191082 >> v1:172.21.15.62:6789/0 conn(0x7f99240cd630 legacy=0x7f99240cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.771+0000 7f9923fff640 1 -- 172.21.15.40:0/1794191082 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992415b8c0 con 0x7f9924154f20 2024-04-25T11:39:16.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.771+0000 7f99237fe640 1 --2- 172.21.15.40:0/1794191082 >> v2:172.21.15.62:3300/0 conn(0x7f9924155300 0x7f99240cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:39:16.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.771+0000 7f9923fff640 1 --2- 172.21.15.40:0/1794191082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 0x7f992415ada0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f99140097a0 tx=0x7f991402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8a8dd464a2151d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.771+0000 7f9922ffd640 1 -- 172.21.15.40:0/1794191082 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914037070 con 0x7f9924154f20 2024-04-25T11:39:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.771+0000 7f9922ffd640 1 -- 172.21.15.40:0/1794191082 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9914032440 con 0x7f9924154f20 2024-04-25T11:39:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.771+0000 7f9922ffd640 1 -- 172.21.15.40:0/1794191082 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914032810 con 0x7f9924154f20 2024-04-25T11:39:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.772+0000 7f992a491640 1 -- 172.21.15.40:0/1794191082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 msgr2=0x7f992415ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.772+0000 7f992a491640 1 --2- 172.21.15.40:0/1794191082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 0x7f992415ada0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f99140097a0 tx=0x7f991402a210 comp rx=0 tx=0).stop 2024-04-25T11:39:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.772+0000 7f992a491640 1 -- 172.21.15.40:0/1794191082 shutdown_connections 2024-04-25T11:39:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.772+0000 7f992a491640 1 --2- 172.21.15.40:0/1794191082 >> v2:172.21.15.62:3300/0 conn(0x7f9924155300 0x7f99240cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.772+0000 7f992a491640 1 --2- 172.21.15.40:0/1794191082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 0x7f992415ada0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.772+0000 7f992a491640 1 -- 172.21.15.40:0/1794191082 >> 172.21.15.40:0/1794191082 conn(0x7f992406c9a0 msgr2=0x7f992406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.772+0000 7f992a491640 1 -- 172.21.15.40:0/1794191082 shutdown_connections 2024-04-25T11:39:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.772+0000 7f992a491640 1 -- 172.21.15.40:0/1794191082 wait complete. 2024-04-25T11:39:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.773+0000 7f992a491640 1 Processor -- start 2024-04-25T11:39:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.773+0000 7f992a491640 1 -- start start 2024-04-25T11:39:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f992a491640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99240cd630 0x7f9924157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f992a491640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 0x7f99240ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f992a491640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9924155300 0x7f99240cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f992a491640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f992415b2e0 con 0x7f9924155300 2024-04-25T11:39:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f992a491640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f99240cce30 con 0x7f99240cd630 2024-04-25T11:39:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f992a491640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f99240c78a0 con 0x7f9924154f20 2024-04-25T11:39:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f9928a2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9924155300 0x7f99240cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f9928a2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9924155300 0x7f99240cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37760/0 (socket says 172.21.15.40:37760) 2024-04-25T11:39:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f9928a2e640 1 -- 172.21.15.40:0/33480929 learned_addr learned my addr 172.21.15.40:0/33480929 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f9923fff640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99240cd630 0x7f9924157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f99237fe640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 0x7f99240ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f9928a2e640 1 -- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 msgr2=0x7f99240ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f9928a2e640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9924154f20 0x7f99240ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f9928a2e640 1 -- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99240cd630 msgr2=0x7f9924157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f9928a2e640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99240cd630 0x7f9924157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.774+0000 7f9928a2e640 1 -- 172.21.15.40:0/33480929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99241582b0 con 0x7f9924155300 2024-04-25T11:39:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.775+0000 7f9928a2e640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9924155300 0x7f99240cfb50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f991c00cbc0 tx=0x7f991c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.775+0000 7f9920ff9640 1 -- 172.21.15.40:0/33480929 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c017070 con 0x7f9924155300 2024-04-25T11:39:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.775+0000 7f9920ff9640 1 -- 172.21.15.40:0/33480929 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991c00ad40 con 0x7f9924155300 2024-04-25T11:39:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.775+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9924158550 con 0x7f9924155300 2024-04-25T11:39:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.775+0000 7f9920ff9640 1 -- 172.21.15.40:0/33480929 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c012600 con 0x7f9924155300 2024-04-25T11:39:16.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.775+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99241f4700 con 0x7f9924155300 2024-04-25T11:39:16.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.776+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f992406e6e0 con 0x7f9924155300 2024-04-25T11:39:16.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.776+0000 7f9920ff9640 1 -- 172.21.15.40:0/33480929 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f991c01a070 con 0x7f9924155300 2024-04-25T11:39:16.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.777+0000 7f9920ff9640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f990406c920 0x7f990406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.777+0000 7f9920ff9640 1 -- 172.21.15.40:0/33480929 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f991c08b720 con 0x7f9924155300 2024-04-25T11:39:16.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.777+0000 7f9923fff640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f990406c920 0x7f990406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.777+0000 7f9920ff9640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7f9904074c40 0x7f99040770c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.777+0000 7f9920ff9640 1 -- 172.21.15.40:0/33480929 --> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9904077790 con 0x7f9904074c40 2024-04-25T11:39:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.777+0000 7f9920ff9640 1 -- 172.21.15.40:0/33480929 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f991c092050 con 0x7f9924155300 2024-04-25T11:39:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.777+0000 7f99237fe640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7f9904074c40 0x7f99040770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.778+0000 7f9923fff640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f990406c920 0x7f990406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9914034000 tx=0x7f991402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.778+0000 7f99237fe640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7f9904074c40 0x7f99040770c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.779+0000 7f9920ff9640 1 -- 172.21.15.40:0/33480929 <== osd.5 v2:172.21.15.62:6824/3210426647 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9904077790 con 0x7f9904074c40 2024-04-25T11:39:16.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 --> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9924073b30 con 0x7f9904074c40 2024-04-25T11:39:16.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f9920ff9640 1 -- 172.21.15.40:0/33480929 <== osd.5 v2:172.21.15.62:6824/3210426647 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9924073b30 con 0x7f9904074c40 2024-04-25T11:39:16.779 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:39:16.779 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:39:16.780 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:39:16.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7f9904074c40 msgr2=0x7f99040770c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f992a491640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6824/3210426647,v1:172.21.15.62:6825/3210426647] conn(0x7f9904074c40 0x7f99040770c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f990406c920 msgr2=0x7f990406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f992a491640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f990406c920 0x7f990406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9914034000 tx=0x7f991402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9924155300 msgr2=0x7f99240cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f992a491640 1 --2- 172.21.15.40:0/33480929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9924155300 0x7f99240cfb50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f991c00cbc0 tx=0x7f991c00a430 comp rx=0 tx=0).stop 2024-04-25T11:39:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f9928a2e640 1 -- 172.21.15.40:0/33480929 reap_dead start 2024-04-25T11:39:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 shutdown_connections 2024-04-25T11:39:16.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.792+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 >> 172.21.15.40:0/33480929 conn(0x7f992406c9a0 msgr2=0x7f992414f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.793+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 shutdown_connections 2024-04-25T11:39:16.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.793+0000 7f992a491640 1 -- 172.21.15.40:0/33480929 wait complete. 2024-04-25T11:39:16.790 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T11:39:16.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.905+0000 7f6bd5c7c640 1 Processor -- start 2024-04-25T11:39:16.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.905+0000 7f6bd5c7c640 1 -- start start 2024-04-25T11:39:16.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bd5c7c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6bd01583d0 0x7f6bd00c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bd5c7c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6bd006ca80 0x7f6bd00ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bd5c7c640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bd00c46a0 con 0x7f6bd006dd00 2024-04-25T11:39:16.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bd5c7c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd00c3bb0 con 0x7f6bd006ca80 2024-04-25T11:39:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bd5c7c640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd01587b0 con 0x7f6bd01583d0 2024-04-25T11:39:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bcffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6bd006ca80 0x7f6bd00ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bcffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6bd006ca80 0x7f6bd00ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37772/0 (socket says 172.21.15.40:37772) 2024-04-25T11:39:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bcffff640 1 -- 172.21.15.40:0/930126159 learned_addr learned my addr 172.21.15.40:0/930126159 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bcf7fe640 1 --2- 172.21.15.40:0/930126159 >> v2:172.21.15.62:3300/0 conn(0x7f6bd01583d0 0x7f6bd00c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bcffff640 1 -- 172.21.15.40:0/930126159 >> v2:172.21.15.62:3300/0 conn(0x7f6bd01583d0 msgr2=0x7f6bd00c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bcffff640 1 --2- 172.21.15.40:0/930126159 >> v2:172.21.15.62:3300/0 conn(0x7f6bd01583d0 0x7f6bd00c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bcffff640 1 -- 172.21.15.40:0/930126159 >> v1:172.21.15.62:6789/0 conn(0x7f6bd006dd00 legacy=0x7f6bd015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.906+0000 7f6bcffff640 1 -- 172.21.15.40:0/930126159 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd00cf250 con 0x7f6bd006ca80 2024-04-25T11:39:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.907+0000 7f6bcffff640 1 --2- 172.21.15.40:0/930126159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd006ca80 0x7f6bd00ce2e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009e50 tx=0x7f6bc802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922b297f56b89ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.907+0000 7f6bce7fc640 1 -- 172.21.15.40:0/930126159 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8036070 con 0x7f6bd006ca80 2024-04-25T11:39:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.907+0000 7f6bce7fc640 1 -- 172.21.15.40:0/930126159 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc8031600 con 0x7f6bd006ca80 2024-04-25T11:39:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.907+0000 7f6bce7fc640 1 -- 172.21.15.40:0/930126159 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80319d0 con 0x7f6bd006ca80 2024-04-25T11:39:16.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.908+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/930126159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd006ca80 msgr2=0x7f6bd00ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.908+0000 7f6bd5c7c640 1 --2- 172.21.15.40:0/930126159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd006ca80 0x7f6bd00ce2e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009e50 tx=0x7f6bc802a760 comp rx=0 tx=0).stop 2024-04-25T11:39:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.908+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/930126159 shutdown_connections 2024-04-25T11:39:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.908+0000 7f6bd5c7c640 1 --2- 172.21.15.40:0/930126159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd006ca80 0x7f6bd00ce2e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.908+0000 7f6bd5c7c640 1 --2- 172.21.15.40:0/930126159 >> v2:172.21.15.62:3300/0 conn(0x7f6bd01583d0 0x7f6bd00c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.908+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/930126159 >> 172.21.15.40:0/930126159 conn(0x7f6bd00713f0 msgr2=0x7f6bd0071800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.908+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/930126159 shutdown_connections 2024-04-25T11:39:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.908+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/930126159 wait complete. 2024-04-25T11:39:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.909+0000 7f6bd5c7c640 1 Processor -- start 2024-04-25T11:39:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.909+0000 7f6bd5c7c640 1 -- start start 2024-04-25T11:39:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bd5c7c640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bd006ca80 0x7f6bd01571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bd5c7c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd006dd00 0x7f6bd015be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bd5c7c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd01583d0 0x7f6bd01f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bd5c7c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd00c43c0 con 0x7f6bd01583d0 2024-04-25T11:39:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bd5c7c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd00c40e0 con 0x7f6bd006dd00 2024-04-25T11:39:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bd5c7c640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd006d500 con 0x7f6bd006ca80 2024-04-25T11:39:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd01583d0 0x7f6bd01f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd01583d0 0x7f6bd01f3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37774/0 (socket says 172.21.15.40:37774) 2024-04-25T11:39:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcffff640 1 -- 172.21.15.40:0/3386410668 learned_addr learned my addr 172.21.15.40:0/3386410668 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bceffd640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd006dd00 0x7f6bd015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcf7fe640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bd006ca80 0x7f6bd01571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcffff640 1 -- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bd006ca80 msgr2=0x7f6bd01571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcffff640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bd006ca80 0x7f6bd01571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcffff640 1 -- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd006dd00 msgr2=0x7f6bd015be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcffff640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd006dd00 0x7f6bd015be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcffff640 1 -- 172.21.15.40:0/3386410668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd00c7a00 con 0x7f6bd01583d0 2024-04-25T11:39:16.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bcffff640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd01583d0 0x7f6bd01f3fb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8034040 tx=0x7f6bc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bc7fff640 1 -- 172.21.15.40:0/3386410668 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8036050 con 0x7f6bd01583d0 2024-04-25T11:39:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bc7fff640 1 -- 172.21.15.40:0/3386410668 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc8042070 con 0x7f6bd01583d0 2024-04-25T11:39:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bc7fff640 1 -- 172.21.15.40:0/3386410668 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8009510 con 0x7f6bd01583d0 2024-04-25T11:39:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.910+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd00c7ce0 con 0x7f6bd01583d0 2024-04-25T11:39:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.911+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd00c82a0 con 0x7f6bd01583d0 2024-04-25T11:39:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.911+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd004fd50 con 0x7f6bd01583d0 2024-04-25T11:39:16.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.912+0000 7f6bc7fff640 1 -- 172.21.15.40:0/3386410668 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6bc8007bf0 con 0x7f6bd01583d0 2024-04-25T11:39:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.912+0000 7f6bc7fff640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6bb006c920 0x7f6bb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.912+0000 7f6bc7fff640 1 -- 172.21.15.40:0/3386410668 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6bc80ac8a0 con 0x7f6bd01583d0 2024-04-25T11:39:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.912+0000 7f6bcf7fe640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6bb006c920 0x7f6bb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.913+0000 7f6bcf7fe640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6bb006c920 0x7f6bb006ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009a30 tx=0x7f6bc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:16.917+0000 7f6bc7fff640 1 -- 172.21.15.40:0/3386410668 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6bc807abd0 con 0x7f6bd01583d0 2024-04-25T11:39:17.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.103+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f6bd0153000 con 0x7f6bb006c920 2024-04-25T11:39:17.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.104+0000 7f6bc7fff640 1 -- 172.21.15.40:0/3386410668 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6bd0153000 con 0x7f6bb006c920 2024-04-25T11:39:17.091 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T11:39:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6bb006c920 msgr2=0x7f6bb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6bb006c920 0x7f6bb006ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009a30 tx=0x7f6bc0008040 comp rx=0 tx=0).stop 2024-04-25T11:39:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd01583d0 msgr2=0x7f6bd01f3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd01583d0 0x7f6bd01f3fb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8034040 tx=0x7f6bc8004a40 comp rx=0 tx=0).stop 2024-04-25T11:39:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 shutdown_connections 2024-04-25T11:39:17.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f6bb006c920 0x7f6bb006ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bd01583d0 0x7f6bd01f3fb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd006dd00 0x7f6bd015be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 --2- 172.21.15.40:0/3386410668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6bd006ca80 0x7f6bd01571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 >> 172.21.15.40:0/3386410668 conn(0x7f6bd00713f0 msgr2=0x7f6bd00c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 shutdown_connections 2024-04-25T11:39:17.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.106+0000 7f6bd5c7c640 1 -- 172.21.15.40:0/3386410668 wait complete. 2024-04-25T11:39:17.104 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T11:39:17.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:39:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.222+0000 7f5f54722640 1 Processor -- start 2024-04-25T11:39:17.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.222+0000 7f5f54722640 1 -- start start 2024-04-25T11:39:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f54722640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f4c0c9660 0x7f5f4c0c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f54722640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f4c0c83c0 0x7f5f4c0c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f54722640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f4c158d00 con 0x7f5f4c0c9280 2024-04-25T11:39:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f54722640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5f4c0c5c90 con 0x7f5f4c0c9660 2024-04-25T11:39:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f54722640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4c0c7ad0 con 0x7f5f4c0c83c0 2024-04-25T11:39:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f524be640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5f4c0c9280 0x7f5f4c0c5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34548/0 (socket says 172.21.15.40:34548) 2024-04-25T11:39:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f51cbd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f4c0c9660 0x7f5f4c0c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f524be640 1 -- 172.21.15.40:0/2235819405 learned_addr learned my addr 172.21.15.40:0/2235819405 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f52cbf640 1 --2- 172.21.15.40:0/2235819405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f4c0c83c0 0x7f5f4c0c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f51cbd640 1 -- 172.21.15.40:0/2235819405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f4c0c83c0 msgr2=0x7f5f4c0c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:17.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f51cbd640 1 --2- 172.21.15.40:0/2235819405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f4c0c83c0 0x7f5f4c0c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f51cbd640 1 -- 172.21.15.40:0/2235819405 >> v1:172.21.15.40:6789/0 conn(0x7f5f4c0c9280 legacy=0x7f5f4c0c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f514bc640 1 -- 172.21.15.40:0/2235819405 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3475612786 0 0) 0x7f5f4c158d00 con 0x7f5f4c0c9280 2024-04-25T11:39:17.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.223+0000 7f5f51cbd640 1 -- 172.21.15.40:0/2235819405 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4c158a20 con 0x7f5f4c0c9660 2024-04-25T11:39:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.224+0000 7f5f51cbd640 1 --2- 172.21.15.40:0/2235819405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9660 0x7f5f4c0c3700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f400097a0 tx=0x7f5f4002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9365eedbc790a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.224+0000 7f5f514bc640 1 -- 172.21.15.40:0/2235819405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40037070 con 0x7f5f4c0c9660 2024-04-25T11:39:17.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.224+0000 7f5f514bc640 1 -- 172.21.15.40:0/2235819405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f40032440 con 0x7f5f4c0c9660 2024-04-25T11:39:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.224+0000 7f5f514bc640 1 -- 172.21.15.40:0/2235819405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40032810 con 0x7f5f4c0c9660 2024-04-25T11:39:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.225+0000 7f5f54722640 1 -- 172.21.15.40:0/2235819405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9660 msgr2=0x7f5f4c0c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.225+0000 7f5f54722640 1 --2- 172.21.15.40:0/2235819405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9660 0x7f5f4c0c3700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f400097a0 tx=0x7f5f4002a210 comp rx=0 tx=0).stop 2024-04-25T11:39:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.225+0000 7f5f54722640 1 -- 172.21.15.40:0/2235819405 shutdown_connections 2024-04-25T11:39:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.225+0000 7f5f54722640 1 --2- 172.21.15.40:0/2235819405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f4c0c83c0 0x7f5f4c0c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.225+0000 7f5f54722640 1 --2- 172.21.15.40:0/2235819405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9660 0x7f5f4c0c3700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.225+0000 7f5f54722640 1 -- 172.21.15.40:0/2235819405 >> 172.21.15.40:0/2235819405 conn(0x7f5f4c06b970 msgr2=0x7f5f4c06bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.225+0000 7f5f54722640 1 -- 172.21.15.40:0/2235819405 shutdown_connections 2024-04-25T11:39:17.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.225+0000 7f5f54722640 1 -- 172.21.15.40:0/2235819405 wait complete. 2024-04-25T11:39:17.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.226+0000 7f5f54722640 1 Processor -- start 2024-04-25T11:39:17.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.226+0000 7f5f54722640 1 -- start start 2024-04-25T11:39:17.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f54722640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c0c83c0 0x7f5f4c1e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f54722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9280 0x7f5f4c1e0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f54722640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f4c1e1f30 0x7f5f4c1e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f54722640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c0c72d0 con 0x7f5f4c0c9280 2024-04-25T11:39:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f54722640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c0c6ff0 con 0x7f5f4c0c83c0 2024-04-25T11:39:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f54722640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4c0c75b0 con 0x7f5f4c1e1f30 2024-04-25T11:39:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f51cbd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9280 0x7f5f4c1e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f51cbd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9280 0x7f5f4c1e0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37794/0 (socket says 172.21.15.40:37794) 2024-04-25T11:39:17.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f51cbd640 1 -- 172.21.15.40:0/658113488 learned_addr learned my addr 172.21.15.40:0/658113488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f52cbf640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f4c1e1f30 0x7f5f4c1e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f51cbd640 1 -- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f4c1e1f30 msgr2=0x7f5f4c1e07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f51cbd640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f4c1e1f30 0x7f5f4c1e07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f51cbd640 1 -- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c0c83c0 msgr2=0x7f5f4c1e3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f524be640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c0c83c0 0x7f5f4c1e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f51cbd640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4c0c83c0 0x7f5f4c1e3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f51cbd640 1 -- 172.21.15.40:0/658113488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4c1e45c0 con 0x7f5f4c0c9280 2024-04-25T11:39:17.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.227+0000 7f5f51cbd640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9280 0x7f5f4c1e0270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5f400097a0 tx=0x7f5f400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.228+0000 7f5f4affd640 1 -- 172.21.15.40:0/658113488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40037050 con 0x7f5f4c0c9280 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.228+0000 7f5f4affd640 1 -- 172.21.15.40:0/658113488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f40038040 con 0x7f5f4c0c9280 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.228+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4c1f1c50 con 0x7f5f4c0c9280 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.228+0000 7f5f4affd640 1 -- 172.21.15.40:0/658113488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40004490 con 0x7f5f4c0c9280 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.228+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f4c1f2210 con 0x7f5f4c0c9280 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.228+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f4c068bb0 con 0x7f5f4c0c9280 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.229+0000 7f5f4affd640 1 -- 172.21.15.40:0/658113488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f4003c6d0 con 0x7f5f4c0c9280 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.229+0000 7f5f4affd640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5f3806c920 0x7f5f3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.229+0000 7f5f4affd640 1 -- 172.21.15.40:0/658113488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5f400ac690 con 0x7f5f4c0c9280 2024-04-25T11:39:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.230+0000 7f5f4affd640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f5f38074ca0 0x7f5f38077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.230+0000 7f5f4affd640 1 -- 172.21.15.40:0/658113488 --> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f4002be50 con 0x7f5f38074ca0 2024-04-25T11:39:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.230+0000 7f5f4affd640 1 -- 172.21.15.40:0/658113488 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5f400b3050 con 0x7f5f4c0c9280 2024-04-25T11:39:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.230+0000 7f5f524be640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5f3806c920 0x7f5f3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.230+0000 7f5f52cbf640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f5f38074ca0 0x7f5f38077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.230+0000 7f5f52cbf640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f5f38074ca0 0x7f5f38077120 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.230+0000 7f5f524be640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5f3806c920 0x7f5f3806ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5f44007fd0 tx=0x7f5f44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.231+0000 7f5f4affd640 1 -- 172.21.15.40:0/658113488 <== osd.6 v2:172.21.15.62:6802/1111875962 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f4002be50 con 0x7f5f38074ca0 2024-04-25T11:39:17.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.244+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 --> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5f4c071820 con 0x7f5f38074ca0 2024-04-25T11:39:17.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f4affd640 1 -- 172.21.15.40:0/658113488 <== osd.6 v2:172.21.15.62:6802/1111875962 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5f4c071820 con 0x7f5f38074ca0 2024-04-25T11:39:17.232 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:39:17.232 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:39:17.232 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:39:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f5f38074ca0 msgr2=0x7f5f38077120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6802/1111875962,v1:172.21.15.62:6804/1111875962] conn(0x7f5f38074ca0 0x7f5f38077120 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5f3806c920 msgr2=0x7f5f3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5f3806c920 0x7f5f3806ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5f44007fd0 tx=0x7f5f44009040 comp rx=0 tx=0).stop 2024-04-25T11:39:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9280 msgr2=0x7f5f4c1e0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 --2- 172.21.15.40:0/658113488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c0c9280 0x7f5f4c1e0270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5f400097a0 tx=0x7f5f400046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f52cbf640 1 -- 172.21.15.40:0/658113488 reap_dead start 2024-04-25T11:39:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 shutdown_connections 2024-04-25T11:39:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 >> 172.21.15.40:0/658113488 conn(0x7f5f4c06b970 msgr2=0x7f5f4c155140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 shutdown_connections 2024-04-25T11:39:17.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.245+0000 7f5f54722640 1 -- 172.21.15.40:0/658113488 wait complete. 2024-04-25T11:39:17.242 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T11:39:17.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.357+0000 7f80404d0640 1 Processor -- start 2024-04-25T11:39:17.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.357+0000 7f80404d0640 1 -- start start 2024-04-25T11:39:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f80404d0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8038154f30 0x7f80380cba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f80404d0640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80380b6180 0x7f80380cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f80404d0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80380d1020 con 0x7f80380c7ab0 2024-04-25T11:39:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f80404d0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f80380cb320 con 0x7f8038154f30 2024-04-25T11:39:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f80404d0640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f803815b820 con 0x7f80380b6180 2024-04-25T11:39:17.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f803e26c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8038154f30 0x7f80380cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f803e26c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8038154f30 0x7f80380cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37800/0 (socket says 172.21.15.40:37800) 2024-04-25T11:39:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f803e26c640 1 -- 172.21.15.40:0/2497529471 learned_addr learned my addr 172.21.15.40:0/2497529471 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f803da6b640 1 --2- 172.21.15.40:0/2497529471 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80380b6180 0x7f80380cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f803e26c640 1 -- 172.21.15.40:0/2497529471 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80380b6180 msgr2=0x7f80380cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f803e26c640 1 --2- 172.21.15.40:0/2497529471 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80380b6180 0x7f80380cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.358+0000 7f803e26c640 1 -- 172.21.15.40:0/2497529471 >> v1:172.21.15.62:6789/0 conn(0x7f80380c7ab0 legacy=0x7f80380c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:17.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.359+0000 7f803e26c640 1 -- 172.21.15.40:0/2497529471 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80380cb5c0 con 0x7f8038154f30 2024-04-25T11:39:17.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.359+0000 7f803e26c640 1 --2- 172.21.15.40:0/2497529471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8038154f30 0x7f80380cba70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8028009e50 tx=0x7f802802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cae0597f735dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.359+0000 7f803d26a640 1 -- 172.21.15.40:0/2497529471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028036070 con 0x7f8038154f30 2024-04-25T11:39:17.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.359+0000 7f803d26a640 1 -- 172.21.15.40:0/2497529471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8028031600 con 0x7f8038154f30 2024-04-25T11:39:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.359+0000 7f803d26a640 1 -- 172.21.15.40:0/2497529471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80280319d0 con 0x7f8038154f30 2024-04-25T11:39:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.360+0000 7f80404d0640 1 -- 172.21.15.40:0/2497529471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8038154f30 msgr2=0x7f80380cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.360+0000 7f80404d0640 1 --2- 172.21.15.40:0/2497529471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8038154f30 0x7f80380cba70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8028009e50 tx=0x7f802802a7a0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.360+0000 7f80404d0640 1 -- 172.21.15.40:0/2497529471 shutdown_connections 2024-04-25T11:39:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.360+0000 7f80404d0640 1 --2- 172.21.15.40:0/2497529471 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80380b6180 0x7f80380cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.360+0000 7f80404d0640 1 --2- 172.21.15.40:0/2497529471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8038154f30 0x7f80380cba70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.360+0000 7f80404d0640 1 -- 172.21.15.40:0/2497529471 >> 172.21.15.40:0/2497529471 conn(0x7f80380d0420 msgr2=0x7f80380d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.360+0000 7f80404d0640 1 -- 172.21.15.40:0/2497529471 shutdown_connections 2024-04-25T11:39:17.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.360+0000 7f80404d0640 1 -- 172.21.15.40:0/2497529471 wait complete. 2024-04-25T11:39:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.361+0000 7f80404d0640 1 Processor -- start 2024-04-25T11:39:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.361+0000 7f80404d0640 1 -- start start 2024-04-25T11:39:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f80404d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80380b6180 0x7f803814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f80404d0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80380c7ab0 0x7f8038159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f80404d0640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8038154f30 0x7f80381538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f80404d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f803815bb00 con 0x7f80380b6180 2024-04-25T11:39:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f80404d0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8038155310 con 0x7f80380c7ab0 2024-04-25T11:39:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f80404d0640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f80380ccae0 con 0x7f8038154f30 2024-04-25T11:39:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803e26c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80380b6180 0x7f803814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803e26c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80380b6180 0x7f803814aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37808/0 (socket says 172.21.15.40:37808) 2024-04-25T11:39:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803e26c640 1 -- 172.21.15.40:0/2395164218 learned_addr learned my addr 172.21.15.40:0/2395164218 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:17.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803ea6d640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8038154f30 0x7f80381538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803da6b640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80380c7ab0 0x7f8038159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803e26c640 1 -- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8038154f30 msgr2=0x7f80381538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803e26c640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8038154f30 0x7f80381538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803e26c640 1 -- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80380c7ab0 msgr2=0x7f8038159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803e26c640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80380c7ab0 0x7f8038159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803e26c640 1 -- 172.21.15.40:0/2395164218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803814d3c0 con 0x7f80380b6180 2024-04-25T11:39:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.362+0000 7f803ea6d640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8038154f30 0x7f80381538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.363+0000 7f803e26c640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80380b6180 0x7f803814aa70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8028009e50 tx=0x7f8028004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.363+0000 7f8036ffd640 1 -- 172.21.15.40:0/2395164218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028041020 con 0x7f80380b6180 2024-04-25T11:39:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.363+0000 7f8036ffd640 1 -- 172.21.15.40:0/2395164218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8028007800 con 0x7f80380b6180 2024-04-25T11:39:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.363+0000 7f8036ffd640 1 -- 172.21.15.40:0/2395164218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028036040 con 0x7f80380b6180 2024-04-25T11:39:17.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.363+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8038158ca0 con 0x7f80380b6180 2024-04-25T11:39:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.363+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8038155710 con 0x7f80380b6180 2024-04-25T11:39:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.364+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f803804fce0 con 0x7f80380b6180 2024-04-25T11:39:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.364+0000 7f8036ffd640 1 -- 172.21.15.40:0/2395164218 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8028044020 con 0x7f80380b6180 2024-04-25T11:39:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.365+0000 7f8036ffd640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f802406c920 0x7f802406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.365+0000 7f8036ffd640 1 -- 172.21.15.40:0/2395164218 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f80280ac860 con 0x7f80380b6180 2024-04-25T11:39:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.365+0000 7f803da6b640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f802406c920 0x7f802406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.366+0000 7f803da6b640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f802406c920 0x7f802406ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f80381f4e40 tx=0x7f802c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.369+0000 7f8036ffd640 1 -- 172.21.15.40:0/2395164218 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f802804b030 con 0x7f80380b6180 2024-04-25T11:39:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.552+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f8038156b10 con 0x7f802406c920 2024-04-25T11:39:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.552+0000 7f8036ffd640 1 -- 172.21.15.40:0/2395164218 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f802c009d80 con 0x7f802406c920 2024-04-25T11:39:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T11:39:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f802406c920 msgr2=0x7f802406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f802406c920 0x7f802406ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f80381f4e40 tx=0x7f802c008040 comp rx=0 tx=0).stop 2024-04-25T11:39:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80380b6180 msgr2=0x7f803814aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80380b6180 0x7f803814aa70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8028009e50 tx=0x7f8028004a40 comp rx=0 tx=0).stop 2024-04-25T11:39:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 shutdown_connections 2024-04-25T11:39:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f802406c920 0x7f802406ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8038154f30 0x7f80381538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80380c7ab0 0x7f8038159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 --2- 172.21.15.40:0/2395164218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80380b6180 0x7f803814aa70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 >> 172.21.15.40:0/2395164218 conn(0x7f80380d0420 msgr2=0x7f80380ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 shutdown_connections 2024-04-25T11:39:17.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.554+0000 7f80404d0640 1 -- 172.21.15.40:0/2395164218 wait complete. 2024-04-25T11:39:17.552 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T11:39:17.552 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:39:17.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.670+0000 7f2322f08640 1 Processor -- start 2024-04-25T11:39:17.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.670+0000 7f2322f08640 1 -- start start 2024-04-25T11:39:17.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f2322f08640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f231c0c7ba0 0x7f231c0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f2322f08640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f231c0cb860 0x7f231c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f2322f08640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f231c0b6420 con 0x7f231c1550a0 2024-04-25T11:39:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f2322f08640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f231c15ac80 con 0x7f231c0cb860 2024-04-25T11:39:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f2322f08640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f231c0cc7b0 con 0x7f231c0c7ba0 2024-04-25T11:39:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f23214a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f231c0cb860 0x7f231c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f23214a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f231c0cb860 0x7f231c15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37814/0 (socket says 172.21.15.40:37814) 2024-04-25T11:39:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f23214a5640 1 -- 172.21.15.40:0/1185226003 learned_addr learned my addr 172.21.15.40:0/1185226003 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f231bfff640 1 --2- 172.21.15.40:0/1185226003 >> v2:172.21.15.62:3300/0 conn(0x7f231c0c7ba0 0x7f231c0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f23214a5640 1 -- 172.21.15.40:0/1185226003 >> v2:172.21.15.62:3300/0 conn(0x7f231c0c7ba0 msgr2=0x7f231c0cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f23214a5640 1 --2- 172.21.15.40:0/1185226003 >> v2:172.21.15.62:3300/0 conn(0x7f231c0c7ba0 0x7f231c0cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.671+0000 7f23214a5640 1 -- 172.21.15.40:0/1185226003 >> v1:172.21.15.62:6789/0 conn(0x7f231c1550a0 legacy=0x7f231c0cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.672+0000 7f23214a5640 1 -- 172.21.15.40:0/1185226003 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c0cca70 con 0x7f231c0cb860 2024-04-25T11:39:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.672+0000 7f23214a5640 1 --2- 172.21.15.40:0/1185226003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0cb860 0x7f231c15e1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f231c06cf30 tx=0x7f230402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1995cfc90ed7df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.672+0000 7f231b7fe640 1 -- 172.21.15.40:0/1185226003 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304037070 con 0x7f231c0cb860 2024-04-25T11:39:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.672+0000 7f231b7fe640 1 -- 172.21.15.40:0/1185226003 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2304032440 con 0x7f231c0cb860 2024-04-25T11:39:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.672+0000 7f231b7fe640 1 -- 172.21.15.40:0/1185226003 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304032810 con 0x7f231c0cb860 2024-04-25T11:39:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.673+0000 7f2322f08640 1 -- 172.21.15.40:0/1185226003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0cb860 msgr2=0x7f231c15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.673+0000 7f2322f08640 1 --2- 172.21.15.40:0/1185226003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0cb860 0x7f231c15e1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f231c06cf30 tx=0x7f230402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.673+0000 7f2322f08640 1 -- 172.21.15.40:0/1185226003 shutdown_connections 2024-04-25T11:39:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.673+0000 7f2322f08640 1 --2- 172.21.15.40:0/1185226003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0cb860 0x7f231c15e1a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.673+0000 7f2322f08640 1 --2- 172.21.15.40:0/1185226003 >> v2:172.21.15.62:3300/0 conn(0x7f231c0c7ba0 0x7f231c0cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.673+0000 7f2322f08640 1 -- 172.21.15.40:0/1185226003 >> 172.21.15.40:0/1185226003 conn(0x7f231c06c9a0 msgr2=0x7f231c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.673+0000 7f2322f08640 1 -- 172.21.15.40:0/1185226003 shutdown_connections 2024-04-25T11:39:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.673+0000 7f2322f08640 1 -- 172.21.15.40:0/1185226003 wait complete. 2024-04-25T11:39:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.674+0000 7f2322f08640 1 Processor -- start 2024-04-25T11:39:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.674+0000 7f2322f08640 1 -- start start 2024-04-25T11:39:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2322f08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0c7ba0 0x7f231c1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2322f08640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f231c0cb860 0x7f231c151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2322f08640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f231c1550a0 0x7f231c14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2322f08640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f231c15b500 con 0x7f231c0c7ba0 2024-04-25T11:39:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2322f08640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f231c15b220 con 0x7f231c0cb860 2024-04-25T11:39:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2322f08640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f231c15b7e0 con 0x7f231c1550a0 2024-04-25T11:39:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2320ca4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0c7ba0 0x7f231c1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2320ca4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0c7ba0 0x7f231c1511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37818/0 (socket says 172.21.15.40:37818) 2024-04-25T11:39:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2320ca4640 1 -- 172.21.15.40:0/150964929 learned_addr learned my addr 172.21.15.40:0/150964929 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f23214a5640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f231c1550a0 0x7f231c14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f231bfff640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f231c0cb860 0x7f231c151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2320ca4640 1 -- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f231c1550a0 msgr2=0x7f231c14f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2320ca4640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f231c1550a0 0x7f231c14f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2320ca4640 1 -- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f231c0cb860 msgr2=0x7f231c151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2320ca4640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f231c0cb860 0x7f231c151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.675+0000 7f2320ca4640 1 -- 172.21.15.40:0/150964929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c150110 con 0x7f231c0c7ba0 2024-04-25T11:39:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.676+0000 7f2320ca4640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0c7ba0 0x7f231c1511b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f231400c760 tx=0x7f231400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.676+0000 7f23197fa640 1 -- 172.21.15.40:0/150964929 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314016020 con 0x7f231c0c7ba0 2024-04-25T11:39:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.676+0000 7f23197fa640 1 -- 172.21.15.40:0/150964929 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2314006d50 con 0x7f231c0c7ba0 2024-04-25T11:39:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.676+0000 7f23197fa640 1 -- 172.21.15.40:0/150964929 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314003bb0 con 0x7f231c0c7ba0 2024-04-25T11:39:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.676+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231c1503f0 con 0x7f231c0c7ba0 2024-04-25T11:39:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.676+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231c1509b0 con 0x7f231c0c7ba0 2024-04-25T11:39:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.677+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f231c068bb0 con 0x7f231c0c7ba0 2024-04-25T11:39:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.677+0000 7f23197fa640 1 -- 172.21.15.40:0/150964929 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f231400a850 con 0x7f231c0c7ba0 2024-04-25T11:39:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.678+0000 7f23197fa640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f22fc06c8f0 0x7f22fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.678+0000 7f23197fa640 1 -- 172.21.15.40:0/150964929 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f231408b640 con 0x7f231c0c7ba0 2024-04-25T11:39:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.678+0000 7f231bfff640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f22fc06c8f0 0x7f22fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.678+0000 7f23197fa640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7f22fc074c70 0x7f22fc0770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.678+0000 7f23197fa640 1 -- 172.21.15.40:0/150964929 --> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f231401eec0 con 0x7f22fc074c70 2024-04-25T11:39:17.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.678+0000 7f23197fa640 1 -- 172.21.15.40:0/150964929 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2314092050 con 0x7f231c0c7ba0 2024-04-25T11:39:17.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.678+0000 7f23214a5640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7f22fc074c70 0x7f22fc0770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.678+0000 7f231bfff640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f22fc06c8f0 0x7f22fc06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f231c0b6000 tx=0x7f230c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.679+0000 7f23214a5640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7f22fc074c70 0x7f22fc0770f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.679+0000 7f23197fa640 1 -- 172.21.15.40:0/150964929 <== osd.7 v2:172.21.15.62:6816/3160032582 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f231401eec0 con 0x7f22fc074c70 2024-04-25T11:39:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.692+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 --> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f231c0cbe80 con 0x7f22fc074c70 2024-04-25T11:39:17.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f23197fa640 1 -- 172.21.15.40:0/150964929 <== osd.7 v2:172.21.15.62:6816/3160032582 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f231c0cbe80 con 0x7f22fc074c70 2024-04-25T11:39:17.680 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T11:39:17.681 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:39:17.681 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7f22fc074c70 msgr2=0x7f22fc0770f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f2322f08640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6816/3160032582,v1:172.21.15.62:6817/3160032582] conn(0x7f22fc074c70 0x7f22fc0770f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f22fc06c8f0 msgr2=0x7f22fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f2322f08640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f22fc06c8f0 0x7f22fc06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f231c0b6000 tx=0x7f230c009190 comp rx=0 tx=0).stop 2024-04-25T11:39:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0c7ba0 msgr2=0x7f231c1511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f2322f08640 1 --2- 172.21.15.40:0/150964929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f231c0c7ba0 0x7f231c1511b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f231400c760 tx=0x7f231400cc20 comp rx=0 tx=0).stop 2024-04-25T11:39:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f23214a5640 1 -- 172.21.15.40:0/150964929 reap_dead start 2024-04-25T11:39:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 shutdown_connections 2024-04-25T11:39:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.693+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 >> 172.21.15.40:0/150964929 conn(0x7f231c06c9a0 msgr2=0x7f231c15e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.694+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 shutdown_connections 2024-04-25T11:39:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.694+0000 7f2322f08640 1 -- 172.21.15.40:0/150964929 wait complete. 2024-04-25T11:39:17.691 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T11:39:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.807+0000 7f1a43144640 1 Processor -- start 2024-04-25T11:39:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.807+0000 7f1a43144640 1 -- start start 2024-04-25T11:39:17.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a43144640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1a3c154e10 0x7f1a3c0c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a43144640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c1551f0 0x7f1a3c0c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a43144640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a3c071ac0 0x7f1a3c0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a43144640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3c0ca030 con 0x7f1a3c154e10 2024-04-25T11:39:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a43144640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3c0c8790 con 0x7f1a3c071ac0 2024-04-25T11:39:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a43144640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3c0c8a50 con 0x7f1a3c1551f0 2024-04-25T11:39:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a40ee0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1a3c154e10 0x7f1a3c0c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a40ee0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1a3c154e10 0x7f1a3c0c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37834/0 (socket says 172.21.15.40:37834) 2024-04-25T11:39:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a40ee0640 1 -- 172.21.15.40:0/818599865 learned_addr learned my addr 172.21.15.40:0/818599865 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a3bfff640 1 --2- 172.21.15.40:0/818599865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c1551f0 0x7f1a3c0c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a416e1640 1 --2- 172.21.15.40:0/818599865 >> v2:172.21.15.62:3300/0 conn(0x7f1a3c071ac0 0x7f1a3c0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a40ee0640 1 -- 172.21.15.40:0/818599865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c1551f0 msgr2=0x7f1a3c0c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a40ee0640 1 --2- 172.21.15.40:0/818599865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c1551f0 0x7f1a3c0c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a40ee0640 1 -- 172.21.15.40:0/818599865 >> v2:172.21.15.62:3300/0 conn(0x7f1a3c071ac0 msgr2=0x7f1a3c0c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a40ee0640 1 --2- 172.21.15.40:0/818599865 >> v2:172.21.15.62:3300/0 conn(0x7f1a3c071ac0 0x7f1a3c0c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.808+0000 7f1a40ee0640 1 -- 172.21.15.40:0/818599865 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c0c8d30 con 0x7f1a3c154e10 2024-04-25T11:39:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.809+0000 7f1a40ee0640 1 --2- 172.21.15.40:0/818599865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 0x7f1a3c0c9af0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0c4770 tx=0x7f1a3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c382db207a3d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.809+0000 7f1a3b7fe640 1 -- 172.21.15.40:0/818599865 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30037070 con 0x7f1a3c154e10 2024-04-25T11:39:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.809+0000 7f1a3b7fe640 1 -- 172.21.15.40:0/818599865 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a30032440 con 0x7f1a3c154e10 2024-04-25T11:39:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.809+0000 7f1a3b7fe640 1 -- 172.21.15.40:0/818599865 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30032810 con 0x7f1a3c154e10 2024-04-25T11:39:17.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.810+0000 7f1a43144640 1 -- 172.21.15.40:0/818599865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 msgr2=0x7f1a3c0c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.810+0000 7f1a43144640 1 --2- 172.21.15.40:0/818599865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 0x7f1a3c0c9af0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0c4770 tx=0x7f1a3002a270 comp rx=0 tx=0).stop 2024-04-25T11:39:17.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.810+0000 7f1a43144640 1 -- 172.21.15.40:0/818599865 shutdown_connections 2024-04-25T11:39:17.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.810+0000 7f1a43144640 1 --2- 172.21.15.40:0/818599865 >> v2:172.21.15.62:3300/0 conn(0x7f1a3c071ac0 0x7f1a3c0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.810+0000 7f1a43144640 1 --2- 172.21.15.40:0/818599865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c1551f0 0x7f1a3c0c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.810+0000 7f1a43144640 1 --2- 172.21.15.40:0/818599865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 0x7f1a3c0c9af0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.810+0000 7f1a43144640 1 -- 172.21.15.40:0/818599865 >> 172.21.15.40:0/818599865 conn(0x7f1a3c06b970 msgr2=0x7f1a3c06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.810+0000 7f1a43144640 1 -- 172.21.15.40:0/818599865 shutdown_connections 2024-04-25T11:39:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.810+0000 7f1a43144640 1 -- 172.21.15.40:0/818599865 wait complete. 2024-04-25T11:39:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.811+0000 7f1a43144640 1 Processor -- start 2024-04-25T11:39:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.811+0000 7f1a43144640 1 -- start start 2024-04-25T11:39:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a43144640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c071ac0 0x7f1a3c1f1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a43144640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 0x7f1a3c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a43144640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a3c159a50 0x7f1a3c15da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a43144640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c0c95d0 con 0x7f1a3c154e10 2024-04-25T11:39:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a43144640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c0c92f0 con 0x7f1a3c159a50 2024-04-25T11:39:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a43144640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3c15bad0 con 0x7f1a3c071ac0 2024-04-25T11:39:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a3bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 0x7f1a3c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a3bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 0x7f1a3c15c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37840/0 (socket says 172.21.15.40:37840) 2024-04-25T11:39:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a3bfff640 1 -- 172.21.15.40:0/3081876426 learned_addr learned my addr 172.21.15.40:0/3081876426 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a40ee0640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c071ac0 0x7f1a3c1f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a416e1640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a3c159a50 0x7f1a3c15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a3bfff640 1 -- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c071ac0 msgr2=0x7f1a3c1f1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a3bfff640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c071ac0 0x7f1a3c1f1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a3bfff640 1 -- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a3c159a50 msgr2=0x7f1a3c15da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a3bfff640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a3c159a50 0x7f1a3c15da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a3bfff640 1 -- 172.21.15.40:0/3081876426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c15d4d0 con 0x7f1a3c154e10 2024-04-25T11:39:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a3bfff640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 0x7f1a3c15c860 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e870 tx=0x7f1a2800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.812+0000 7f1a397fa640 1 -- 172.21.15.40:0/3081876426 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28017070 con 0x7f1a3c154e10 2024-04-25T11:39:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.813+0000 7f1a397fa640 1 -- 172.21.15.40:0/3081876426 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2800c940 con 0x7f1a3c154e10 2024-04-25T11:39:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.813+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c15cf10 con 0x7f1a3c154e10 2024-04-25T11:39:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.813+0000 7f1a397fa640 1 -- 172.21.15.40:0/3081876426 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800cd30 con 0x7f1a3c154e10 2024-04-25T11:39:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.813+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c1f0920 con 0x7f1a3c154e10 2024-04-25T11:39:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.814+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3c04fd50 con 0x7f1a3c154e10 2024-04-25T11:39:17.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.814+0000 7f1a397fa640 1 -- 172.21.15.40:0/3081876426 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a2801a070 con 0x7f1a3c154e10 2024-04-25T11:39:17.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.814+0000 7f1a397fa640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1a2006c920 0x7f1a2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.814+0000 7f1a397fa640 1 -- 172.21.15.40:0/3081876426 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1a2808bd70 con 0x7f1a3c154e10 2024-04-25T11:39:17.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.815+0000 7f1a40ee0640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1a2006c920 0x7f1a2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.815+0000 7f1a40ee0640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1a2006c920 0x7f1a2006ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a30034000 tx=0x7f1a3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:17.819+0000 7f1a397fa640 1 -- 172.21.15.40:0/3081876426 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a28059a40 con 0x7f1a3c154e10 2024-04-25T11:39:17.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.003+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f1a3c0cd0f0 con 0x7f1a2006c920 2024-04-25T11:39:17.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.004+0000 7f1a397fa640 1 -- 172.21.15.40:0/3081876426 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1a3c0cd0f0 con 0x7f1a2006c920 2024-04-25T11:39:17.991 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T11:39:17.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1a2006c920 msgr2=0x7f1a2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1a2006c920 0x7f1a2006ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a30034000 tx=0x7f1a3002e000 comp rx=0 tx=0).stop 2024-04-25T11:39:17.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 msgr2=0x7f1a3c15c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 0x7f1a3c15c860 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e870 tx=0x7f1a2800ed30 comp rx=0 tx=0).stop 2024-04-25T11:39:17.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 shutdown_connections 2024-04-25T11:39:17.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1a2006c920 0x7f1a2006ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a3c159a50 0x7f1a3c15da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1a3c154e10 0x7f1a3c15c860 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 --2- 172.21.15.40:0/3081876426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a3c071ac0 0x7f1a3c1f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 >> 172.21.15.40:0/3081876426 conn(0x7f1a3c06b970 msgr2=0x7f1a3c071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 shutdown_connections 2024-04-25T11:39:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.006+0000 7f1a43144640 1 -- 172.21.15.40:0/3081876426 wait complete. 2024-04-25T11:39:18.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:39:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.119+0000 7fcb49459640 1 Processor -- start 2024-04-25T11:39:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.119+0000 7fcb49459640 1 -- start start 2024-04-25T11:39:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb49459640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb440c8c60 0x7fcb440c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb49459640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb440c9610 con 0x7fcb4406fbc0 2024-04-25T11:39:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb49459640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb44158c60 con 0x7fcb441530b0 2024-04-25T11:39:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb49459640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcb44159b50 con 0x7fcb440c8c60 2024-04-25T11:39:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb437fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb440c8c60 0x7fcb440c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb3bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcb4406fbc0 0x7fcb440c5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34550/0 (socket says 172.21.15.40:34550) 2024-04-25T11:39:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb3bfff640 1 -- 172.21.15.40:0/2245583828 learned_addr learned my addr 172.21.15.40:0/2245583828 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb437fe640 1 -- 172.21.15.40:0/2245583828 >> v1:172.21.15.62:6789/0 conn(0x7fcb441530b0 legacy=0x7fcb440c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb427fc640 1 -- 172.21.15.40:0/2245583828 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3839094435 0 0) 0x7fcb440c9610 con 0x7fcb4406fbc0 2024-04-25T11:39:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb437fe640 1 -- 172.21.15.40:0/2245583828 >> v1:172.21.15.40:6789/0 conn(0x7fcb4406fbc0 legacy=0x7fcb440c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.120+0000 7fcb437fe640 1 -- 172.21.15.40:0/2245583828 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb44158f40 con 0x7fcb440c8c60 2024-04-25T11:39:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.121+0000 7fcb437fe640 1 --2- 172.21.15.40:0/2245583828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 0x7fcb440c90d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009d90 tx=0x7fcb3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c9794c1950a6ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.121+0000 7fcb427fc640 1 -- 172.21.15.40:0/2245583828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c007540 con 0x7fcb440c8c60 2024-04-25T11:39:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.121+0000 7fcb427fc640 1 -- 172.21.15.40:0/2245583828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c0076e0 con 0x7fcb440c8c60 2024-04-25T11:39:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.121+0000 7fcb427fc640 1 -- 172.21.15.40:0/2245583828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c007ab0 con 0x7fcb440c8c60 2024-04-25T11:39:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.122+0000 7fcb49459640 1 -- 172.21.15.40:0/2245583828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 msgr2=0x7fcb440c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.122+0000 7fcb49459640 1 --2- 172.21.15.40:0/2245583828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 0x7fcb440c90d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009d90 tx=0x7fcb3c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.122+0000 7fcb49459640 1 -- 172.21.15.40:0/2245583828 shutdown_connections 2024-04-25T11:39:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.122+0000 7fcb49459640 1 --2- 172.21.15.40:0/2245583828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 0x7fcb440c90d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.122+0000 7fcb49459640 1 -- 172.21.15.40:0/2245583828 >> 172.21.15.40:0/2245583828 conn(0x7fcb4406b970 msgr2=0x7fcb4406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.122+0000 7fcb49459640 1 -- 172.21.15.40:0/2245583828 shutdown_connections 2024-04-25T11:39:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.122+0000 7fcb49459640 1 -- 172.21.15.40:0/2245583828 wait complete. 2024-04-25T11:39:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.123+0000 7fcb49459640 1 Processor -- start 2024-04-25T11:39:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.123+0000 7fcb49459640 1 -- start start 2024-04-25T11:39:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb49459640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb4406fbc0 0x7fcb440c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb49459640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 0x7fcb440bd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb49459640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb441530b0 0x7fcb440bd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb49459640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441597e0 con 0x7fcb440c8c60 2024-04-25T11:39:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb49459640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44159500 con 0x7fcb441530b0 2024-04-25T11:39:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb49459640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcb440c7310 con 0x7fcb4406fbc0 2024-04-25T11:39:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 0x7fcb440bd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 0x7fcb440bd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37858/0 (socket says 172.21.15.40:37858) 2024-04-25T11:39:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb437fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb441530b0 0x7fcb440bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3bfff640 1 -- 172.21.15.40:0/726416731 learned_addr learned my addr 172.21.15.40:0/726416731 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb42ffd640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb4406fbc0 0x7fcb440c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3bfff640 1 -- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb4406fbc0 msgr2=0x7fcb440c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3bfff640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb4406fbc0 0x7fcb440c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3bfff640 1 -- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb441530b0 msgr2=0x7fcb440bd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3bfff640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb441530b0 0x7fcb440bd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3bfff640 1 -- 172.21.15.40:0/726416731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb44152520 con 0x7fcb440c8c60 2024-04-25T11:39:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3bfff640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 0x7fcb440bd320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00c3f0 tx=0x7fcb2c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3b7fe640 1 -- 172.21.15.40:0/726416731 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0074e0 con 0x7fcb440c8c60 2024-04-25T11:39:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3b7fe640 1 -- 172.21.15.40:0/726416731 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb2c0047e0 con 0x7fcb440c8c60 2024-04-25T11:39:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb3b7fe640 1 -- 172.21.15.40:0/726416731 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c004c50 con 0x7fcb440c8c60 2024-04-25T11:39:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.124+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb440bf120 con 0x7fcb440c8c60 2024-04-25T11:39:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.125+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb44151a20 con 0x7fcb440c8c60 2024-04-25T11:39:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.125+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb44068bb0 con 0x7fcb440c8c60 2024-04-25T11:39:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.126+0000 7fcb3b7fe640 1 -- 172.21.15.40:0/726416731 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb2c011070 con 0x7fcb440c8c60 2024-04-25T11:39:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.126+0000 7fcb3b7fe640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb2406c8f0 0x7fcb2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.126+0000 7fcb3b7fe640 1 -- 172.21.15.40:0/726416731 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fcb2c08b5c0 con 0x7fcb440c8c60 2024-04-25T11:39:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.127+0000 7fcb42ffd640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb2406c8f0 0x7fcb2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.127+0000 7fcb42ffd640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb2406c8f0 0x7fcb2406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcb34000f60 tx=0x7fcb34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.131+0000 7fcb3b7fe640 1 -- 172.21.15.40:0/726416731 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb2c059320 con 0x7fcb440c8c60 2024-04-25T11:39:18.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.312+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb4414fda0 con 0x7fcb2406c8f0 2024-04-25T11:39:18.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.319+0000 7fcb3b7fe640 1 -- 172.21.15.40:0/726416731 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251860 (secure 0 0 0) 0x7fcb4414fda0 con 0x7fcb2406c8f0 2024-04-25T11:39:18.309 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:18.310 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:39:18.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb2406c8f0 msgr2=0x7fcb2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb2406c8f0 0x7fcb2406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcb34000f60 tx=0x7fcb34009040 comp rx=0 tx=0).stop 2024-04-25T11:39:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 msgr2=0x7fcb440bd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 0x7fcb440bd320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00c3f0 tx=0x7fcb2c00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 shutdown_connections 2024-04-25T11:39:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fcb2406c8f0 0x7fcb2406edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb441530b0 0x7fcb440bd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb440c8c60 0x7fcb440bd320 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 --2- 172.21.15.40:0/726416731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb4406fbc0 0x7fcb440c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 >> 172.21.15.40:0/726416731 conn(0x7fcb4406b970 msgr2=0x7fcb440ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 shutdown_connections 2024-04-25T11:39:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:18.322+0000 7fcb49459640 1 -- 172.21.15.40:0/726416731 wait complete. 2024-04-25T11:39:18.320 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-04-25T11:39:16.960776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601545,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":377,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218144,"kb_used_data":2008,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756576,"statfs":{"total":773094113280,"available":772870733824,"internally_reserved":0,"allocated":2056192,"data_stored":1782844,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001786"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119398+0000","last_change":"2024-04-25T11:38:25.677572+0000","last_active":"2024-04-25T11:39:13.119398+0000","last_peered":"2024-04-25T11:39:13.119398+0000","last_clean":"2024-04-25T11:39:13.119398+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:39:13.119398+0000","last_undegraded":"2024-04-25T11:39:13.119398+0000","last_fullsized":"2024-04-25T11:39:13.119398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108764+0000","last_change":"2024-04-25T11:38:25.677437+0000","last_active":"2024-04-25T11:39:13.108764+0000","last_peered":"2024-04-25T11:39:13.108764+0000","last_clean":"2024-04-25T11:39:13.108764+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:39:13.108764+0000","last_undegraded":"2024-04-25T11:39:13.108764+0000","last_fullsized":"2024-04-25T11:39:13.108764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.200954+0000","last_change":"2024-04-25T11:38:25.676553+0000","last_active":"2024-04-25T11:39:13.200954+0000","last_peered":"2024-04-25T11:39:13.200954+0000","last_clean":"2024-04-25T11:39:13.200954+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:39:13.200954+0000","last_undegraded":"2024-04-25T11:39:13.200954+0000","last_fullsized":"2024-04-25T11:39:13.200954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.110698+0000","last_change":"2024-04-25T11:38:25.676518+0000","last_active":"2024-04-25T11:39:13.110698+0000","last_peered":"2024-04-25T11:39:13.110698+0000","last_clean":"2024-04-25T11:39:13.110698+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:39:13.110698+0000","last_undegraded":"2024-04-25T11:39:13.110698+0000","last_fullsized":"2024-04-25T11:39:13.110698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.118958+0000","last_change":"2024-04-25T11:38:25.678524+0000","last_active":"2024-04-25T11:39:13.118958+0000","last_peered":"2024-04-25T11:39:13.118958+0000","last_clean":"2024-04-25T11:39:13.118958+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:39:13.118958+0000","last_undegraded":"2024-04-25T11:39:13.118958+0000","last_fullsized":"2024-04-25T11:39:13.118958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.110932+0000","last_change":"2024-04-25T11:38:25.683983+0000","last_active":"2024-04-25T11:39:13.110932+0000","last_peered":"2024-04-25T11:39:13.110932+0000","last_clean":"2024-04-25T11:39:13.110932+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:39:13.110932+0000","last_undegraded":"2024-04-25T11:39:13.110932+0000","last_fullsized":"2024-04-25T11:39:13.110932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.118762+0000","last_change":"2024-04-25T11:38:25.682904+0000","last_active":"2024-04-25T11:39:13.118762+0000","last_peered":"2024-04-25T11:39:13.118762+0000","last_clean":"2024-04-25T11:39:13.118762+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:39:13.118762+0000","last_undegraded":"2024-04-25T11:39:13.118762+0000","last_fullsized":"2024-04-25T11:39:13.118762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"peering","last_fresh":"2024-04-25T11:38:28.590897+0000","last_change":"2024-04-25T11:39:13.113553+0000","last_active":"2024-04-25T11:39:13.113553+0000","last_peered":"2024-04-25T11:38:28.590897+0000","last_clean":"2024-04-25T11:38:28.590897+0000","last_became_active":"2024-04-25T11:38:25.669988+0000","last_became_peered":"2024-04-25T11:38:25.669988+0000","last_unstale":"2024-04-25T11:38:28.590897+0000","last_undegraded":"2024-04-25T11:38:28.590897+0000","last_fullsized":"2024-04-25T11:38:28.590897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119464+0000","last_change":"2024-04-25T11:38:25.678459+0000","last_active":"2024-04-25T11:39:13.119464+0000","last_peered":"2024-04-25T11:39:13.119464+0000","last_clean":"2024-04-25T11:39:13.119464+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:39:13.119464+0000","last_undegraded":"2024-04-25T11:39:13.119464+0000","last_fullsized":"2024-04-25T11:39:13.119464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:00.321164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112967+0000","last_change":"2024-04-25T11:38:25.675561+0000","last_active":"2024-04-25T11:39:13.112967+0000","last_peered":"2024-04-25T11:39:13.112967+0000","last_clean":"2024-04-25T11:39:13.112967+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:39:13.112967+0000","last_undegraded":"2024-04-25T11:39:13.112967+0000","last_fullsized":"2024-04-25T11:39:13.112967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112758+0000","last_change":"2024-04-25T11:38:25.671113+0000","last_active":"2024-04-25T11:39:13.112758+0000","last_peered":"2024-04-25T11:39:13.112758+0000","last_clean":"2024-04-25T11:39:13.112758+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:39:13.112758+0000","last_undegraded":"2024-04-25T11:39:13.112758+0000","last_fullsized":"2024-04-25T11:39:13.112758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.121421+0000","last_change":"2024-04-25T11:38:25.665690+0000","last_active":"2024-04-25T11:39:13.121421+0000","last_peered":"2024-04-25T11:39:13.121421+0000","last_clean":"2024-04-25T11:39:13.121421+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:39:13.121421+0000","last_undegraded":"2024-04-25T11:39:13.121421+0000","last_fullsized":"2024-04-25T11:39:13.121421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:25.083420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.113154+0000","last_change":"2024-04-25T11:38:25.666858+0000","last_active":"2024-04-25T11:39:13.113154+0000","last_peered":"2024-04-25T11:39:13.113154+0000","last_clean":"2024-04-25T11:39:13.113154+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:39:13.113154+0000","last_undegraded":"2024-04-25T11:39:13.113154+0000","last_fullsized":"2024-04-25T11:39:13.113154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.200681+0000","last_change":"2024-04-25T11:38:25.677561+0000","last_active":"2024-04-25T11:39:13.200681+0000","last_peered":"2024-04-25T11:39:13.200681+0000","last_clean":"2024-04-25T11:39:13.200681+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:39:13.200681+0000","last_undegraded":"2024-04-25T11:39:13.200681+0000","last_fullsized":"2024-04-25T11:39:13.200681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108431+0000","last_change":"2024-04-25T11:38:25.672129+0000","last_active":"2024-04-25T11:39:13.108431+0000","last_peered":"2024-04-25T11:39:13.108431+0000","last_clean":"2024-04-25T11:39:13.108431+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:39:13.108431+0000","last_undegraded":"2024-04-25T11:39:13.108431+0000","last_fullsized":"2024-04-25T11:39:13.108431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.110470+0000","last_change":"2024-04-25T11:38:25.682547+0000","last_active":"2024-04-25T11:39:13.110470+0000","last_peered":"2024-04-25T11:39:13.110470+0000","last_clean":"2024-04-25T11:39:13.110470+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:39:13.110470+0000","last_undegraded":"2024-04-25T11:39:13.110470+0000","last_fullsized":"2024-04-25T11:39:13.110470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":2,"reported_epoch":21,"state":"peering","last_fresh":"2024-04-25T11:38:28.592558+0000","last_change":"2024-04-25T11:39:13.597247+0000","last_active":"2024-04-25T11:39:13.597247+0000","last_peered":"2024-04-25T11:38:28.592558+0000","last_clean":"2024-04-25T11:38:28.592558+0000","last_became_active":"2024-04-25T11:38:25.682590+0000","last_became_peered":"2024-04-25T11:38:25.682590+0000","last_unstale":"2024-04-25T11:38:28.592558+0000","last_undegraded":"2024-04-25T11:38:28.592558+0000","last_fullsized":"2024-04-25T11:38:28.592558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.117344+0000","last_change":"2024-04-25T11:38:25.669650+0000","last_active":"2024-04-25T11:39:13.117344+0000","last_peered":"2024-04-25T11:39:13.117344+0000","last_clean":"2024-04-25T11:39:13.117344+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:39:13.117344+0000","last_undegraded":"2024-04-25T11:39:13.117344+0000","last_fullsized":"2024-04-25T11:39:13.117344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:23.795745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119347+0000","last_change":"2024-04-25T11:38:25.678238+0000","last_active":"2024-04-25T11:39:13.119347+0000","last_peered":"2024-04-25T11:39:13.119347+0000","last_clean":"2024-04-25T11:39:13.119347+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:39:13.119347+0000","last_undegraded":"2024-04-25T11:39:13.119347+0000","last_fullsized":"2024-04-25T11:39:13.119347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:25.636818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112739+0000","last_change":"2024-04-25T11:38:25.666736+0000","last_active":"2024-04-25T11:39:13.112739+0000","last_peered":"2024-04-25T11:39:13.112739+0000","last_clean":"2024-04-25T11:39:13.112739+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:39:13.112739+0000","last_undegraded":"2024-04-25T11:39:13.112739+0000","last_fullsized":"2024-04-25T11:39:13.112739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.200896+0000","last_change":"2024-04-25T11:38:25.680587+0000","last_active":"2024-04-25T11:39:13.200896+0000","last_peered":"2024-04-25T11:39:13.200896+0000","last_clean":"2024-04-25T11:39:13.200896+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:39:13.200896+0000","last_undegraded":"2024-04-25T11:39:13.200896+0000","last_fullsized":"2024-04-25T11:39:13.200896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.110932+0000","last_change":"2024-04-25T11:38:25.680464+0000","last_active":"2024-04-25T11:39:13.110932+0000","last_peered":"2024-04-25T11:39:13.110932+0000","last_clean":"2024-04-25T11:39:13.110932+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:39:13.110932+0000","last_undegraded":"2024-04-25T11:39:13.110932+0000","last_fullsized":"2024-04-25T11:39:13.110932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":2,"reported_epoch":21,"state":"peering","last_fresh":"2024-04-25T11:38:28.590940+0000","last_change":"2024-04-25T11:39:13.597191+0000","last_active":"2024-04-25T11:39:13.597191+0000","last_peered":"2024-04-25T11:38:28.590940+0000","last_clean":"2024-04-25T11:38:28.590940+0000","last_became_active":"2024-04-25T11:38:25.671920+0000","last_became_peered":"2024-04-25T11:38:25.671920+0000","last_unstale":"2024-04-25T11:38:28.590940+0000","last_undegraded":"2024-04-25T11:38:28.590940+0000","last_fullsized":"2024-04-25T11:38:28.590940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.110614+0000","last_change":"2024-04-25T11:38:25.674108+0000","last_active":"2024-04-25T11:39:13.110614+0000","last_peered":"2024-04-25T11:39:13.110614+0000","last_clean":"2024-04-25T11:39:13.110614+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:39:13.110614+0000","last_undegraded":"2024-04-25T11:39:13.110614+0000","last_fullsized":"2024-04-25T11:39:13.110614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.113076+0000","last_change":"2024-04-25T11:38:25.672215+0000","last_active":"2024-04-25T11:39:13.113076+0000","last_peered":"2024-04-25T11:39:13.113076+0000","last_clean":"2024-04-25T11:39:13.113076+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:39:13.113076+0000","last_undegraded":"2024-04-25T11:39:13.113076+0000","last_fullsized":"2024-04-25T11:39:13.113076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112992+0000","last_change":"2024-04-25T11:38:25.679772+0000","last_active":"2024-04-25T11:39:13.112992+0000","last_peered":"2024-04-25T11:39:13.112992+0000","last_clean":"2024-04-25T11:39:13.112992+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:39:13.112992+0000","last_undegraded":"2024-04-25T11:39:13.112992+0000","last_fullsized":"2024-04-25T11:39:13.112992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108618+0000","last_change":"2024-04-25T11:38:25.665626+0000","last_active":"2024-04-25T11:39:13.108618+0000","last_peered":"2024-04-25T11:39:13.108618+0000","last_clean":"2024-04-25T11:39:13.108618+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:39:13.108618+0000","last_undegraded":"2024-04-25T11:39:13.108618+0000","last_fullsized":"2024-04-25T11:39:13.108618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.113204+0000","last_change":"2024-04-25T11:38:25.672935+0000","last_active":"2024-04-25T11:39:13.113204+0000","last_peered":"2024-04-25T11:39:13.113204+0000","last_clean":"2024-04-25T11:39:13.113204+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:39:13.113204+0000","last_undegraded":"2024-04-25T11:39:13.113204+0000","last_fullsized":"2024-04-25T11:39:13.113204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.200932+0000","last_change":"2024-04-25T11:38:25.679486+0000","last_active":"2024-04-25T11:39:13.200932+0000","last_peered":"2024-04-25T11:39:13.200932+0000","last_clean":"2024-04-25T11:39:13.200932+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:39:13.200932+0000","last_undegraded":"2024-04-25T11:39:13.200932+0000","last_fullsized":"2024-04-25T11:39:13.200932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.643145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.111072+0000","last_change":"2024-04-25T11:38:25.679437+0000","last_active":"2024-04-25T11:39:13.111072+0000","last_peered":"2024-04-25T11:39:13.111072+0000","last_clean":"2024-04-25T11:39:13.111072+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:39:13.111072+0000","last_undegraded":"2024-04-25T11:39:13.111072+0000","last_fullsized":"2024-04-25T11:39:13.111072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.117302+0000","last_change":"2024-04-25T11:38:25.670384+0000","last_active":"2024-04-25T11:39:13.117302+0000","last_peered":"2024-04-25T11:39:13.117302+0000","last_clean":"2024-04-25T11:39:13.117302+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:39:13.117302+0000","last_undegraded":"2024-04-25T11:39:13.117302+0000","last_fullsized":"2024-04-25T11:39:13.117302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112191+0000","last_change":"2024-04-25T11:38:25.680252+0000","last_active":"2024-04-25T11:39:13.112191+0000","last_peered":"2024-04-25T11:39:13.112191+0000","last_clean":"2024-04-25T11:39:13.112191+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:39:13.112191+0000","last_undegraded":"2024-04-25T11:39:13.112191+0000","last_fullsized":"2024-04-25T11:39:13.112191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.121148+0000","last_change":"2024-04-25T11:38:25.669542+0000","last_active":"2024-04-25T11:39:13.121148+0000","last_peered":"2024-04-25T11:39:13.121148+0000","last_clean":"2024-04-25T11:39:13.121148+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:39:13.121148+0000","last_undegraded":"2024-04-25T11:39:13.121148+0000","last_fullsized":"2024-04-25T11:39:13.121148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:51.929330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'7","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.113340+0000","last_change":"2024-04-25T11:38:24.666197+0000","last_active":"2024-04-25T11:39:13.113340+0000","last_peered":"2024-04-25T11:39:13.113340+0000","last_clean":"2024-04-25T11:39:13.113340+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:39:13.113340+0000","last_undegraded":"2024-04-25T11:39:13.113340+0000","last_fullsized":"2024-04-25T11:39:13.113340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:06.198972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.201030+0000","last_change":"2024-04-25T11:38:25.682931+0000","last_active":"2024-04-25T11:39:13.201030+0000","last_peered":"2024-04-25T11:39:13.201030+0000","last_clean":"2024-04-25T11:39:13.201030+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:39:13.201030+0000","last_undegraded":"2024-04-25T11:39:13.201030+0000","last_fullsized":"2024-04-25T11:39:13.201030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:39.220794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112555+0000","last_change":"2024-04-25T11:38:24.660888+0000","last_active":"2024-04-25T11:39:13.112555+0000","last_peered":"2024-04-25T11:39:13.112555+0000","last_clean":"2024-04-25T11:39:13.112555+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:39:13.112555+0000","last_undegraded":"2024-04-25T11:39:13.112555+0000","last_fullsized":"2024-04-25T11:39:13.112555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112865+0000","last_change":"2024-04-25T11:38:24.657815+0000","last_active":"2024-04-25T11:39:13.112865+0000","last_peered":"2024-04-25T11:39:13.112865+0000","last_clean":"2024-04-25T11:39:13.112865+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:39:13.112865+0000","last_undegraded":"2024-04-25T11:39:13.112865+0000","last_fullsized":"2024-04-25T11:39:13.112865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.113801+0000","last_change":"2024-04-25T11:38:25.680969+0000","last_active":"2024-04-25T11:39:13.113801+0000","last_peered":"2024-04-25T11:39:13.113801+0000","last_clean":"2024-04-25T11:39:13.113801+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:39:13.113801+0000","last_undegraded":"2024-04-25T11:39:13.113801+0000","last_fullsized":"2024-04-25T11:39:13.113801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.110872+0000","last_change":"2024-04-25T11:38:24.668899+0000","last_active":"2024-04-25T11:39:13.110872+0000","last_peered":"2024-04-25T11:39:13.110872+0000","last_clean":"2024-04-25T11:39:13.110872+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:39:13.110872+0000","last_undegraded":"2024-04-25T11:39:13.110872+0000","last_fullsized":"2024-04-25T11:39:13.110872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108386+0000","last_change":"2024-04-25T11:38:25.666511+0000","last_active":"2024-04-25T11:39:13.108386+0000","last_peered":"2024-04-25T11:39:13.108386+0000","last_clean":"2024-04-25T11:39:13.108386+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:39:13.108386+0000","last_undegraded":"2024-04-25T11:39:13.108386+0000","last_fullsized":"2024-04-25T11:39:13.108386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'6","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112883+0000","last_change":"2024-04-25T11:38:24.666735+0000","last_active":"2024-04-25T11:39:13.112883+0000","last_peered":"2024-04-25T11:39:13.112883+0000","last_clean":"2024-04-25T11:39:13.112883+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:39:13.112883+0000","last_undegraded":"2024-04-25T11:39:13.112883+0000","last_fullsized":"2024-04-25T11:39:13.112883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.111694+0000","last_change":"2024-04-25T11:38:25.680065+0000","last_active":"2024-04-25T11:39:13.111694+0000","last_peered":"2024-04-25T11:39:13.111694+0000","last_clean":"2024-04-25T11:39:13.111694+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:39:13.111694+0000","last_undegraded":"2024-04-25T11:39:13.111694+0000","last_fullsized":"2024-04-25T11:39:13.111694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:09.694681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.111839+0000","last_change":"2024-04-25T11:38:24.671566+0000","last_active":"2024-04-25T11:39:13.111839+0000","last_peered":"2024-04-25T11:39:13.111839+0000","last_clean":"2024-04-25T11:39:13.111839+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:39:13.111839+0000","last_undegraded":"2024-04-25T11:39:13.111839+0000","last_fullsized":"2024-04-25T11:39:13.111839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.201090+0000","last_change":"2024-04-25T11:38:25.678075+0000","last_active":"2024-04-25T11:39:13.201090+0000","last_peered":"2024-04-25T11:39:13.201090+0000","last_clean":"2024-04-25T11:39:13.201090+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:39:13.201090+0000","last_undegraded":"2024-04-25T11:39:13.201090+0000","last_fullsized":"2024-04-25T11:39:13.201090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108715+0000","last_change":"2024-04-25T11:38:24.667282+0000","last_active":"2024-04-25T11:39:13.108715+0000","last_peered":"2024-04-25T11:39:13.108715+0000","last_clean":"2024-04-25T11:39:13.108715+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:39:13.108715+0000","last_undegraded":"2024-04-25T11:39:13.108715+0000","last_fullsized":"2024-04-25T11:39:13.108715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.121521+0000","last_change":"2024-04-25T11:38:25.680417+0000","last_active":"2024-04-25T11:39:13.121521+0000","last_peered":"2024-04-25T11:39:13.121521+0000","last_clean":"2024-04-25T11:39:13.121521+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:39:13.121521+0000","last_undegraded":"2024-04-25T11:39:13.121521+0000","last_fullsized":"2024-04-25T11:39:13.121521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:49.822692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'14","reported_seq":30,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112744+0000","last_change":"2024-04-25T11:38:24.662562+0000","last_active":"2024-04-25T11:39:13.112744+0000","last_peered":"2024-04-25T11:39:13.112744+0000","last_clean":"2024-04-25T11:39:13.112744+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:39:13.112744+0000","last_undegraded":"2024-04-25T11:39:13.112744+0000","last_fullsized":"2024-04-25T11:39:13.112744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119051+0000","last_change":"2024-04-25T11:38:25.677518+0000","last_active":"2024-04-25T11:39:13.119051+0000","last_peered":"2024-04-25T11:39:13.119051+0000","last_clean":"2024-04-25T11:39:13.119051+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:39:13.119051+0000","last_undegraded":"2024-04-25T11:39:13.119051+0000","last_fullsized":"2024-04-25T11:39:13.119051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.238224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.201035+0000","last_change":"2024-04-25T11:38:24.658960+0000","last_active":"2024-04-25T11:39:13.201035+0000","last_peered":"2024-04-25T11:39:13.201035+0000","last_clean":"2024-04-25T11:39:13.201035+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:39:13.201035+0000","last_undegraded":"2024-04-25T11:39:13.201035+0000","last_fullsized":"2024-04-25T11:39:13.201035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.200811+0000","last_change":"2024-04-25T11:38:25.681355+0000","last_active":"2024-04-25T11:39:13.200811+0000","last_peered":"2024-04-25T11:39:13.200811+0000","last_clean":"2024-04-25T11:39:13.200811+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:39:13.200811+0000","last_undegraded":"2024-04-25T11:39:13.200811+0000","last_fullsized":"2024-04-25T11:39:13.200811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:50.483290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.111115+0000","last_change":"2024-04-25T11:38:24.666747+0000","last_active":"2024-04-25T11:39:13.111115+0000","last_peered":"2024-04-25T11:39:13.111115+0000","last_clean":"2024-04-25T11:39:13.111115+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:39:13.111115+0000","last_undegraded":"2024-04-25T11:39:13.111115+0000","last_fullsized":"2024-04-25T11:39:13.111115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119039+0000","last_change":"2024-04-25T11:38:25.684128+0000","last_active":"2024-04-25T11:39:13.119039+0000","last_peered":"2024-04-25T11:39:13.119039+0000","last_clean":"2024-04-25T11:39:13.119039+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:39:13.119039+0000","last_undegraded":"2024-04-25T11:39:13.119039+0000","last_fullsized":"2024-04-25T11:39:13.119039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'7","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112475+0000","last_change":"2024-04-25T11:38:24.661103+0000","last_active":"2024-04-25T11:39:13.112475+0000","last_peered":"2024-04-25T11:39:13.112475+0000","last_clean":"2024-04-25T11:39:13.112475+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:39:13.112475+0000","last_undegraded":"2024-04-25T11:39:13.112475+0000","last_fullsized":"2024-04-25T11:39:13.112475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:15.677917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112331+0000","last_change":"2024-04-25T11:38:25.680490+0000","last_active":"2024-04-25T11:39:13.112331+0000","last_peered":"2024-04-25T11:39:13.112331+0000","last_clean":"2024-04-25T11:39:13.112331+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:39:13.112331+0000","last_undegraded":"2024-04-25T11:39:13.112331+0000","last_fullsized":"2024-04-25T11:39:13.112331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":33,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.117097+0000","last_change":"2024-04-25T11:38:24.665238+0000","last_active":"2024-04-25T11:39:13.117097+0000","last_peered":"2024-04-25T11:39:13.117097+0000","last_clean":"2024-04-25T11:39:13.117097+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:39:13.117097+0000","last_undegraded":"2024-04-25T11:39:13.117097+0000","last_fullsized":"2024-04-25T11:39:13.117097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.118900+0000","last_change":"2024-04-25T11:38:25.682669+0000","last_active":"2024-04-25T11:39:13.118900+0000","last_peered":"2024-04-25T11:39:13.118900+0000","last_clean":"2024-04-25T11:39:13.118900+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:39:13.118900+0000","last_undegraded":"2024-04-25T11:39:13.118900+0000","last_fullsized":"2024-04-25T11:39:13.118900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:32.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.121571+0000","last_change":"2024-04-25T11:38:24.658666+0000","last_active":"2024-04-25T11:39:13.121571+0000","last_peered":"2024-04-25T11:39:13.121571+0000","last_clean":"2024-04-25T11:39:13.121571+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:39:13.121571+0000","last_undegraded":"2024-04-25T11:39:13.121571+0000","last_fullsized":"2024-04-25T11:39:13.121571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:24.295273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119349+0000","last_change":"2024-04-25T11:38:25.680783+0000","last_active":"2024-04-25T11:39:13.119349+0000","last_peered":"2024-04-25T11:39:13.119349+0000","last_clean":"2024-04-25T11:39:13.119349+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:39:13.119349+0000","last_undegraded":"2024-04-25T11:39:13.119349+0000","last_fullsized":"2024-04-25T11:39:13.119349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.117580+0000","last_change":"2024-04-25T11:38:24.656937+0000","last_active":"2024-04-25T11:39:13.117580+0000","last_peered":"2024-04-25T11:39:13.117580+0000","last_clean":"2024-04-25T11:39:13.117580+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:39:13.117580+0000","last_undegraded":"2024-04-25T11:39:13.117580+0000","last_fullsized":"2024-04-25T11:39:13.117580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119497+0000","last_change":"2024-04-25T11:38:25.681307+0000","last_active":"2024-04-25T11:39:13.119497+0000","last_peered":"2024-04-25T11:39:13.119497+0000","last_clean":"2024-04-25T11:39:13.119497+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:39:13.119497+0000","last_undegraded":"2024-04-25T11:39:13.119497+0000","last_fullsized":"2024-04-25T11:39:13.119497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:17.678183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108819+0000","last_change":"2024-04-25T11:38:24.663247+0000","last_active":"2024-04-25T11:39:13.108819+0000","last_peered":"2024-04-25T11:39:13.108819+0000","last_clean":"2024-04-25T11:39:13.108819+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:39:13.108819+0000","last_undegraded":"2024-04-25T11:39:13.108819+0000","last_fullsized":"2024-04-25T11:39:13.108819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.111379+0000","last_change":"2024-04-25T11:38:25.676409+0000","last_active":"2024-04-25T11:39:13.111379+0000","last_peered":"2024-04-25T11:39:13.111379+0000","last_clean":"2024-04-25T11:39:13.111379+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:39:13.111379+0000","last_undegraded":"2024-04-25T11:39:13.111379+0000","last_fullsized":"2024-04-25T11:39:13.111379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'7","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.117551+0000","last_change":"2024-04-25T11:38:24.666800+0000","last_active":"2024-04-25T11:39:13.117551+0000","last_peered":"2024-04-25T11:39:13.117551+0000","last_clean":"2024-04-25T11:39:13.117551+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:39:13.117551+0000","last_undegraded":"2024-04-25T11:39:13.117551+0000","last_fullsized":"2024-04-25T11:39:13.117551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119200+0000","last_change":"2024-04-25T11:38:25.678136+0000","last_active":"2024-04-25T11:39:13.119200+0000","last_peered":"2024-04-25T11:39:13.119200+0000","last_clean":"2024-04-25T11:39:13.119200+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:39:13.119200+0000","last_undegraded":"2024-04-25T11:39:13.119200+0000","last_fullsized":"2024-04-25T11:39:13.119200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:33.649619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":171,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.118877+0000","last_change":"2024-04-25T11:38:20.865911+0000","last_active":"2024-04-25T11:39:13.118877+0000","last_peered":"2024-04-25T11:39:13.118877+0000","last_clean":"2024-04-25T11:39:13.118877+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:39:13.118877+0000","last_undegraded":"2024-04-25T11:39:13.118877+0000","last_fullsized":"2024-04-25T11:39:13.118877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:19.606342+0000","last_clean_scrub_stamp":"2024-04-25T11:38:19.606342+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:58.126842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.110615+0000","last_change":"2024-04-25T11:38:24.671283+0000","last_active":"2024-04-25T11:39:13.110615+0000","last_peered":"2024-04-25T11:39:13.110615+0000","last_clean":"2024-04-25T11:39:13.110615+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:39:13.110615+0000","last_undegraded":"2024-04-25T11:39:13.110615+0000","last_fullsized":"2024-04-25T11:39:13.110615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.113453+0000","last_change":"2024-04-25T11:38:25.675002+0000","last_active":"2024-04-25T11:39:13.113453+0000","last_peered":"2024-04-25T11:39:13.113453+0000","last_clean":"2024-04-25T11:39:13.113453+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:39:13.113453+0000","last_undegraded":"2024-04-25T11:39:13.113453+0000","last_fullsized":"2024-04-25T11:39:13.113453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.044287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.118716+0000","last_change":"2024-04-25T11:38:24.671213+0000","last_active":"2024-04-25T11:39:13.118716+0000","last_peered":"2024-04-25T11:39:13.118716+0000","last_clean":"2024-04-25T11:39:13.118716+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:39:13.118716+0000","last_undegraded":"2024-04-25T11:39:13.118716+0000","last_fullsized":"2024-04-25T11:39:13.118716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.200731+0000","last_change":"2024-04-25T11:38:25.673353+0000","last_active":"2024-04-25T11:39:13.200731+0000","last_peered":"2024-04-25T11:39:13.200731+0000","last_clean":"2024-04-25T11:39:13.200731+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:39:13.200731+0000","last_undegraded":"2024-04-25T11:39:13.200731+0000","last_fullsized":"2024-04-25T11:39:13.200731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:48.553601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'6","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.117545+0000","last_change":"2024-04-25T11:38:24.650599+0000","last_active":"2024-04-25T11:39:13.117545+0000","last_peered":"2024-04-25T11:39:13.117545+0000","last_clean":"2024-04-25T11:39:13.117545+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:39:13.117545+0000","last_undegraded":"2024-04-25T11:39:13.117545+0000","last_fullsized":"2024-04-25T11:39:13.117545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108418+0000","last_change":"2024-04-25T11:38:25.668956+0000","last_active":"2024-04-25T11:39:13.108418+0000","last_peered":"2024-04-25T11:39:13.108418+0000","last_clean":"2024-04-25T11:39:13.108418+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:39:13.108418+0000","last_undegraded":"2024-04-25T11:39:13.108418+0000","last_fullsized":"2024-04-25T11:39:13.108418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.111207+0000","last_change":"2024-04-25T11:38:24.665496+0000","last_active":"2024-04-25T11:39:13.111207+0000","last_peered":"2024-04-25T11:39:13.111207+0000","last_clean":"2024-04-25T11:39:13.111207+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:39:13.111207+0000","last_undegraded":"2024-04-25T11:39:13.111207+0000","last_fullsized":"2024-04-25T11:39:13.111207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.121408+0000","last_change":"2024-04-25T11:38:25.681800+0000","last_active":"2024-04-25T11:39:13.121408+0000","last_peered":"2024-04-25T11:39:13.121408+0000","last_clean":"2024-04-25T11:39:13.121408+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:39:13.121408+0000","last_undegraded":"2024-04-25T11:39:13.121408+0000","last_fullsized":"2024-04-25T11:39:13.121408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:39.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108596+0000","last_change":"2024-04-25T11:38:24.659792+0000","last_active":"2024-04-25T11:39:13.108596+0000","last_peered":"2024-04-25T11:39:13.108596+0000","last_clean":"2024-04-25T11:39:13.108596+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:39:13.108596+0000","last_undegraded":"2024-04-25T11:39:13.108596+0000","last_fullsized":"2024-04-25T11:39:13.108596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":2,"reported_epoch":21,"state":"peering","last_fresh":"2024-04-25T11:38:25.662843+0000","last_change":"2024-04-25T11:39:13.597225+0000","last_active":"2024-04-25T11:38:24.640920+0000","last_peered":"2024-04-25T11:38:24.640920+0000","last_clean":"2024-04-25T11:38:24.640920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:38:25.662843+0000","last_undegraded":"2024-04-25T11:38:25.662843+0000","last_fullsized":"2024-04-25T11:38:25.662843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119444+0000","last_change":"2024-04-25T11:38:24.671412+0000","last_active":"2024-04-25T11:39:13.119444+0000","last_peered":"2024-04-25T11:39:13.119444+0000","last_clean":"2024-04-25T11:39:13.119444+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:39:13.119444+0000","last_undegraded":"2024-04-25T11:39:13.119444+0000","last_fullsized":"2024-04-25T11:39:13.119444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:34.607841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108488+0000","last_change":"2024-04-25T11:38:25.668959+0000","last_active":"2024-04-25T11:39:13.108488+0000","last_peered":"2024-04-25T11:39:13.108488+0000","last_clean":"2024-04-25T11:39:13.108488+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:39:13.108488+0000","last_undegraded":"2024-04-25T11:39:13.108488+0000","last_fullsized":"2024-04-25T11:39:13.108488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108845+0000","last_change":"2024-04-25T11:38:24.664146+0000","last_active":"2024-04-25T11:39:13.108845+0000","last_peered":"2024-04-25T11:39:13.108845+0000","last_clean":"2024-04-25T11:39:13.108845+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:39:13.108845+0000","last_undegraded":"2024-04-25T11:39:13.108845+0000","last_fullsized":"2024-04-25T11:39:13.108845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:24.079032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.113176+0000","last_change":"2024-04-25T11:38:25.680105+0000","last_active":"2024-04-25T11:39:13.113176+0000","last_peered":"2024-04-25T11:39:13.113176+0000","last_clean":"2024-04-25T11:39:13.113176+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:39:13.113176+0000","last_undegraded":"2024-04-25T11:39:13.113176+0000","last_fullsized":"2024-04-25T11:39:13.113176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:33.341830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112956+0000","last_change":"2024-04-25T11:38:24.668369+0000","last_active":"2024-04-25T11:39:13.112956+0000","last_peered":"2024-04-25T11:39:13.112956+0000","last_clean":"2024-04-25T11:39:13.112956+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:39:13.112956+0000","last_undegraded":"2024-04-25T11:39:13.112956+0000","last_fullsized":"2024-04-25T11:39:13.112956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.200898+0000","last_change":"2024-04-25T11:38:25.676446+0000","last_active":"2024-04-25T11:39:13.200898+0000","last_peered":"2024-04-25T11:39:13.200898+0000","last_clean":"2024-04-25T11:39:13.200898+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:39:13.200898+0000","last_undegraded":"2024-04-25T11:39:13.200898+0000","last_fullsized":"2024-04-25T11:39:13.200898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.118703+0000","last_change":"2024-04-25T11:38:24.671951+0000","last_active":"2024-04-25T11:39:13.118703+0000","last_peered":"2024-04-25T11:39:13.118703+0000","last_clean":"2024-04-25T11:39:13.118703+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:39:13.118703+0000","last_undegraded":"2024-04-25T11:39:13.118703+0000","last_fullsized":"2024-04-25T11:39:13.118703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.924942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.121696+0000","last_change":"2024-04-25T11:38:25.669474+0000","last_active":"2024-04-25T11:39:13.121696+0000","last_peered":"2024-04-25T11:39:13.121696+0000","last_clean":"2024-04-25T11:39:13.121696+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:39:13.121696+0000","last_undegraded":"2024-04-25T11:39:13.121696+0000","last_fullsized":"2024-04-25T11:39:13.121696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:38.359208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108501+0000","last_change":"2024-04-25T11:38:24.668689+0000","last_active":"2024-04-25T11:39:13.108501+0000","last_peered":"2024-04-25T11:39:13.108501+0000","last_clean":"2024-04-25T11:39:13.108501+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:39:13.108501+0000","last_undegraded":"2024-04-25T11:39:13.108501+0000","last_fullsized":"2024-04-25T11:39:13.108501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.113894+0000","last_change":"2024-04-25T11:38:25.668088+0000","last_active":"2024-04-25T11:39:13.113894+0000","last_peered":"2024-04-25T11:39:13.113894+0000","last_clean":"2024-04-25T11:39:13.113894+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:39:13.113894+0000","last_undegraded":"2024-04-25T11:39:13.113894+0000","last_fullsized":"2024-04-25T11:39:13.113894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:21.629454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.108742+0000","last_change":"2024-04-25T11:38:24.662913+0000","last_active":"2024-04-25T11:39:13.108742+0000","last_peered":"2024-04-25T11:39:13.108742+0000","last_clean":"2024-04-25T11:39:13.108742+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:39:13.108742+0000","last_undegraded":"2024-04-25T11:39:13.108742+0000","last_fullsized":"2024-04-25T11:39:13.108742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.214999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.113250+0000","last_change":"2024-04-25T11:38:25.680244+0000","last_active":"2024-04-25T11:39:13.113250+0000","last_peered":"2024-04-25T11:39:13.113250+0000","last_clean":"2024-04-25T11:39:13.113250+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:39:13.113250+0000","last_undegraded":"2024-04-25T11:39:13.113250+0000","last_fullsized":"2024-04-25T11:39:13.113250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:19.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119273+0000","last_change":"2024-04-25T11:38:24.669196+0000","last_active":"2024-04-25T11:39:13.119273+0000","last_peered":"2024-04-25T11:39:13.119273+0000","last_clean":"2024-04-25T11:39:13.119273+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:39:13.119273+0000","last_undegraded":"2024-04-25T11:39:13.119273+0000","last_fullsized":"2024-04-25T11:39:13.119273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:22.791358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112496+0000","last_change":"2024-04-25T11:38:25.668400+0000","last_active":"2024-04-25T11:39:13.112496+0000","last_peered":"2024-04-25T11:39:13.112496+0000","last_clean":"2024-04-25T11:39:13.112496+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:39:13.112496+0000","last_undegraded":"2024-04-25T11:39:13.112496+0000","last_fullsized":"2024-04-25T11:39:13.112496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:08.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.111372+0000","last_change":"2024-04-25T11:38:25.683663+0000","last_active":"2024-04-25T11:39:13.111372+0000","last_peered":"2024-04-25T11:39:13.111372+0000","last_clean":"2024-04-25T11:39:13.111372+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:39:13.111372+0000","last_undegraded":"2024-04-25T11:39:13.111372+0000","last_fullsized":"2024-04-25T11:39:13.111372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:17.758724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119078+0000","last_change":"2024-04-25T11:38:24.670671+0000","last_active":"2024-04-25T11:39:13.119078+0000","last_peered":"2024-04-25T11:39:13.119078+0000","last_clean":"2024-04-25T11:39:13.119078+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:39:13.119078+0000","last_undegraded":"2024-04-25T11:39:13.119078+0000","last_fullsized":"2024-04-25T11:39:13.119078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.367661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.110841+0000","last_change":"2024-04-25T11:38:25.681439+0000","last_active":"2024-04-25T11:39:13.110841+0000","last_peered":"2024-04-25T11:39:13.110841+0000","last_clean":"2024-04-25T11:39:13.110841+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:39:13.110841+0000","last_undegraded":"2024-04-25T11:39:13.110841+0000","last_fullsized":"2024-04-25T11:39:13.110841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":28,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.119233+0000","last_change":"2024-04-25T11:38:24.669485+0000","last_active":"2024-04-25T11:39:13.119233+0000","last_peered":"2024-04-25T11:39:13.119233+0000","last_clean":"2024-04-25T11:39:13.119233+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:39:13.119233+0000","last_undegraded":"2024-04-25T11:39:13.119233+0000","last_fullsized":"2024-04-25T11:39:13.119233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:48.818304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.200973+0000","last_change":"2024-04-25T11:38:25.684415+0000","last_active":"2024-04-25T11:39:13.200973+0000","last_peered":"2024-04-25T11:39:13.200973+0000","last_clean":"2024-04-25T11:39:13.200973+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:39:13.200973+0000","last_undegraded":"2024-04-25T11:39:13.200973+0000","last_fullsized":"2024-04-25T11:39:13.200973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:33.740020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112957+0000","last_change":"2024-04-25T11:38:24.658741+0000","last_active":"2024-04-25T11:39:13.112957+0000","last_peered":"2024-04-25T11:39:13.112957+0000","last_clean":"2024-04-25T11:39:13.112957+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:39:13.112957+0000","last_undegraded":"2024-04-25T11:39:13.112957+0000","last_fullsized":"2024-04-25T11:39:13.112957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.112854+0000","last_change":"2024-04-25T11:38:25.667505+0000","last_active":"2024-04-25T11:39:13.112854+0000","last_peered":"2024-04-25T11:39:13.112854+0000","last_clean":"2024-04-25T11:39:13.112854+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:39:13.112854+0000","last_undegraded":"2024-04-25T11:39:13.112854+0000","last_fullsized":"2024-04-25T11:39:13.112854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:03.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:39:13.201099+0000","last_change":"2024-04-25T11:38:24.669215+0000","last_active":"2024-04-25T11:39:13.201099+0000","last_peered":"2024-04-25T11:39:13.201099+0000","last_clean":"2024-04-25T11:39:13.201099+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:39:13.201099+0000","last_undegraded":"2024-04-25T11:39:13.201099+0000","last_fullsized":"2024-04-25T11:39:13.201099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:23.636532+0000","last_clean_scrub_stamp":"2024-04-25T11:38:23.636532+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.902257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11177,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":218,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":22354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192,"ondisk_log_size":192,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":319488,"data_stored":670125,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":245760,"data_stored":81456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":75448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":75349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":299008,"data_stored":662176,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:39:18.326 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.326 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.326 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.326 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.327 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.327 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.327 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.327 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.327 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.327 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.328 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.328 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.328 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.328 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.328 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.328 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.328 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.329 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.329 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.329 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.329 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.329 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.329 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.330 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.330 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.330 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.330 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.330 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.330 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.331 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.331 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.331 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.331 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.331 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.331 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.331 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.332 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.332 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.332 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.332 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.332 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.332 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.332 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.333 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.333 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.333 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.333 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.333 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.333 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.334 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.334 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.334 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.334 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.334 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.334 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.334 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.335 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.335 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.335 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.335 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.335 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.335 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.336 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.336 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.336 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T11:38:19.606342+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=19, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.336 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.336 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.336 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.336 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.337 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.337 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.337 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.337 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.337 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.337 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.338 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.338 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.338 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.338 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.338 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.338 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.338 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.339 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.339 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.339 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.339 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.339 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.339 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.340 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.340 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.340 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.340 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.340 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.340 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.340 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.341 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.341 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-04-25T11:38:23.636532+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:18.341 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T11:39:27.765 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:39:27.769 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:39:38.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:39:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.468+0000 7f1b48937640 1 Processor -- start 2024-04-25T11:39:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.468+0000 7f1b48937640 1 -- start start 2024-04-25T11:39:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.469+0000 7f1b48937640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1b44151bc0 0x7f1b44158500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.469+0000 7f1b48937640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.469+0000 7f1b48937640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b44072c80 con 0x7f1b44151fa0 2024-04-25T11:39:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.469+0000 7f1b48937640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f1b44158a40 con 0x7f1b44151bc0 2024-04-25T11:39:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.469+0000 7f1b48937640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1b440c3930 con 0x7f1b440722d0 2024-04-25T11:39:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b42d9d640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b4259c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1b44151bc0 0x7f1b44158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b41d9b640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1b44151fa0 0x7f1b440c5510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:52404/0 (socket says 172.21.15.40:52404) 2024-04-25T11:39:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b41d9b640 1 -- 172.21.15.40:0/1873900838 learned_addr learned my addr 172.21.15.40:0/1873900838 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b42d9d640 1 --2- 172.21.15.40:0/1873900838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44072740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:52170/0 (socket says 172.21.15.40:52170) 2024-04-25T11:39:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b42d9d640 1 -- 172.21.15.40:0/1873900838 >> v2:172.21.15.62:3300/0 conn(0x7f1b44151bc0 msgr2=0x7f1b44158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b42d9d640 1 --2- 172.21.15.40:0/1873900838 >> v2:172.21.15.62:3300/0 conn(0x7f1b44151bc0 0x7f1b44158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b42d9d640 1 -- 172.21.15.40:0/1873900838 >> v1:172.21.15.62:6789/0 conn(0x7f1b44151fa0 legacy=0x7f1b440c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b42d9d640 1 -- 172.21.15.40:0/1873900838 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b440c3c10 con 0x7f1b440722d0 2024-04-25T11:39:38.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.470+0000 7f1b4259c640 1 --2- 172.21.15.40:0/1873900838 >> v2:172.21.15.62:3300/0 conn(0x7f1b44151bc0 0x7f1b44158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:39:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.471+0000 7f1b42d9d640 1 --2- 172.21.15.40:0/1873900838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44072740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009d90 tx=0x7f1b3402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1a45eaaf634ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.471+0000 7f1b4159a640 1 -- 172.21.15.40:0/1873900838 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34007740 con 0x7f1b440722d0 2024-04-25T11:39:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.471+0000 7f1b4159a640 1 -- 172.21.15.40:0/1873900838 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b340078e0 con 0x7f1b440722d0 2024-04-25T11:39:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.471+0000 7f1b4159a640 1 -- 172.21.15.40:0/1873900838 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34007cb0 con 0x7f1b440722d0 2024-04-25T11:39:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.471+0000 7f1b48937640 1 -- 172.21.15.40:0/1873900838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 msgr2=0x7f1b44072740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.471+0000 7f1b48937640 1 --2- 172.21.15.40:0/1873900838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44072740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009d90 tx=0x7f1b3402a660 comp rx=0 tx=0).stop 2024-04-25T11:39:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.472+0000 7f1b48937640 1 -- 172.21.15.40:0/1873900838 shutdown_connections 2024-04-25T11:39:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.472+0000 7f1b48937640 1 --2- 172.21.15.40:0/1873900838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44072740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.472+0000 7f1b48937640 1 --2- 172.21.15.40:0/1873900838 >> v2:172.21.15.62:3300/0 conn(0x7f1b44151bc0 0x7f1b44158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.472+0000 7f1b48937640 1 -- 172.21.15.40:0/1873900838 >> 172.21.15.40:0/1873900838 conn(0x7f1b4406b970 msgr2=0x7f1b4406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.472+0000 7f1b48937640 1 -- 172.21.15.40:0/1873900838 shutdown_connections 2024-04-25T11:39:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.472+0000 7f1b48937640 1 -- 172.21.15.40:0/1873900838 wait complete. 2024-04-25T11:39:38.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.473+0000 7f1b48937640 1 Processor -- start 2024-04-25T11:39:38.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.473+0000 7f1b48937640 1 -- start start 2024-04-25T11:39:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b48937640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b48937640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b44151bc0 0x7f1b44154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b48937640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44151fa0 0x7f1b440c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b48937640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44071db0 con 0x7f1b44151bc0 2024-04-25T11:39:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b48937640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44071ad0 con 0x7f1b44151fa0 2024-04-25T11:39:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b48937640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1b440c3ef0 con 0x7f1b440722d0 2024-04-25T11:39:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b41d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b44151bc0 0x7f1b44154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b41d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b44151bc0 0x7f1b44154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35678/0 (socket says 172.21.15.40:35678) 2024-04-25T11:39:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b41d9b640 1 -- 172.21.15.40:0/964231820 learned_addr learned my addr 172.21.15.40:0/964231820 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b42d9d640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44151fa0 0x7f1b440c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b4259c640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b41d9b640 1 -- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 msgr2=0x7f1b44154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b41d9b640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b41d9b640 1 -- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44151fa0 msgr2=0x7f1b440c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b41d9b640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44151fa0 0x7f1b440c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.474+0000 7f1b41d9b640 1 -- 172.21.15.40:0/964231820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b441f1050 con 0x7f1b44151bc0 2024-04-25T11:39:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.475+0000 7f1b41d9b640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b44151bc0 0x7f1b44154e20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c3c0 tx=0x7f1b3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.475+0000 7f1b32ffd640 1 -- 172.21.15.40:0/964231820 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38017070 con 0x7f1b44151bc0 2024-04-25T11:39:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.475+0000 7f1b32ffd640 1 -- 172.21.15.40:0/964231820 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38005ce0 con 0x7f1b44151bc0 2024-04-25T11:39:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.475+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b44158200 con 0x7f1b44151bc0 2024-04-25T11:39:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.475+0000 7f1b32ffd640 1 -- 172.21.15.40:0/964231820 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b380072f0 con 0x7f1b44151bc0 2024-04-25T11:39:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.475+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b441587c0 con 0x7f1b44151bc0 2024-04-25T11:39:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.476+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4414d310 con 0x7f1b44151bc0 2024-04-25T11:39:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.476+0000 7f1b32ffd640 1 -- 172.21.15.40:0/964231820 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b3801a070 con 0x7f1b44151bc0 2024-04-25T11:39:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.477+0000 7f1b32ffd640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1b2406c920 0x7f1b2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.477+0000 7f1b32ffd640 1 -- 172.21.15.40:0/964231820 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1b3808afa0 con 0x7f1b44151bc0 2024-04-25T11:39:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.477+0000 7f1b4259c640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1b2406c920 0x7f1b2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.477+0000 7f1b4259c640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1b2406c920 0x7f1b2406ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009560 tx=0x7f1b3c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.481+0000 7f1b32ffd640 1 -- 172.21.15.40:0/964231820 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b4414d310 con 0x7f1b44151bc0 2024-04-25T11:39:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.664+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b44068bb0 con 0x7f1b2406c920 2024-04-25T11:39:38.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.671+0000 7f1b32ffd640 1 -- 172.21.15.40:0/964231820 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252026 (secure 0 0 0) 0x7f1b44068bb0 con 0x7f1b2406c920 2024-04-25T11:39:38.662 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:38.663 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:39:38.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1b2406c920 msgr2=0x7f1b2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1b2406c920 0x7f1b2406ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009560 tx=0x7f1b3c009190 comp rx=0 tx=0).stop 2024-04-25T11:39:38.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b44151bc0 msgr2=0x7f1b44154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b44151bc0 0x7f1b44154e20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c3c0 tx=0x7f1b3800c880 comp rx=0 tx=0).stop 2024-04-25T11:39:38.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 shutdown_connections 2024-04-25T11:39:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f1b2406c920 0x7f1b2406ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b44151fa0 0x7f1b440c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b44151bc0 0x7f1b44154e20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 --2- 172.21.15.40:0/964231820 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b440722d0 0x7f1b44154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.674+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 >> 172.21.15.40:0/964231820 conn(0x7f1b4406b970 msgr2=0x7f1b44147ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:38.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.675+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 shutdown_connections 2024-04-25T11:39:38.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:38.675+0000 7f1b48937640 1 -- 172.21.15.40:0/964231820 wait complete. 2024-04-25T11:39:38.674 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-04-25T11:39:36.966663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601545,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":377,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218304,"kb_used_data":2176,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756416,"statfs":{"total":773094113280,"available":772870569984,"internally_reserved":0,"allocated":2228224,"data_stored":1868018,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003616"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.614289+0000","last_change":"2024-04-25T11:39:19.614312+0000","last_active":"2024-04-25T11:39:19.614289+0000","last_peered":"2024-04-25T11:39:19.614289+0000","last_clean":"2024-04-25T11:39:19.614289+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:39:19.614289+0000","last_undegraded":"2024-04-25T11:39:19.614289+0000","last_fullsized":"2024-04-25T11:39:19.614289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:19.614191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:19.614191+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.614191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:51.506396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.034115+0000","last_change":"2024-04-25T11:39:27.034141+0000","last_active":"2024-04-25T11:39:27.034115+0000","last_peered":"2024-04-25T11:39:27.034115+0000","last_clean":"2024-04-25T11:39:27.034115+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:39:27.034115+0000","last_undegraded":"2024-04-25T11:39:27.034115+0000","last_fullsized":"2024-04-25T11:39:27.034115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.034025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.034025+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.034025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:20.290265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.936941+0000","last_change":"2024-04-25T11:39:18.936967+0000","last_active":"2024-04-25T11:39:18.936941+0000","last_peered":"2024-04-25T11:39:18.936941+0000","last_clean":"2024-04-25T11:39:18.936941+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:39:18.936941+0000","last_undegraded":"2024-04-25T11:39:18.936941+0000","last_fullsized":"2024-04-25T11:39:18.936941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:18.936865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:18.936865+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.936865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:44.880593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.922517+0000","last_change":"2024-04-25T11:39:24.922539+0000","last_active":"2024-04-25T11:39:24.922517+0000","last_peered":"2024-04-25T11:39:24.922517+0000","last_clean":"2024-04-25T11:39:24.922517+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:39:24.922517+0000","last_undegraded":"2024-04-25T11:39:24.922517+0000","last_fullsized":"2024-04-25T11:39:24.922517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.922439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.922439+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.922439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:32.468649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:30.702804+0000","last_change":"2024-04-25T11:39:30.702842+0000","last_active":"2024-04-25T11:39:30.702804+0000","last_peered":"2024-04-25T11:39:30.702804+0000","last_clean":"2024-04-25T11:39:30.702804+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:39:30.702804+0000","last_undegraded":"2024-04-25T11:39:30.702804+0000","last_fullsized":"2024-04-25T11:39:30.702804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:30.702702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:30.702702+0000","last_clean_scrub_stamp":"2024-04-25T11:39:30.702702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:05.090666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:31.745951+0000","last_change":"2024-04-25T11:39:31.745976+0000","last_active":"2024-04-25T11:39:31.745951+0000","last_peered":"2024-04-25T11:39:31.745951+0000","last_clean":"2024-04-25T11:39:31.745951+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:39:31.745951+0000","last_undegraded":"2024-04-25T11:39:31.745951+0000","last_fullsized":"2024-04-25T11:39:31.745951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:31.745860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:31.745860+0000","last_clean_scrub_stamp":"2024-04-25T11:39:31.745860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:06.982572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:14.574066+0000","last_change":"2024-04-25T11:38:25.682904+0000","last_active":"2024-04-25T11:39:14.574066+0000","last_peered":"2024-04-25T11:39:14.574066+0000","last_clean":"2024-04-25T11:39:14.574066+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:39:14.574066+0000","last_undegraded":"2024-04-25T11:39:14.574066+0000","last_fullsized":"2024-04-25T11:39:14.574066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:28.974546+0000","last_change":"2024-04-25T11:39:28.974567+0000","last_active":"2024-04-25T11:39:28.974546+0000","last_peered":"2024-04-25T11:39:28.974546+0000","last_clean":"2024-04-25T11:39:28.974546+0000","last_became_active":"2024-04-25T11:39:14.979698+0000","last_became_peered":"2024-04-25T11:39:14.979698+0000","last_unstale":"2024-04-25T11:39:28.974546+0000","last_undegraded":"2024-04-25T11:39:28.974546+0000","last_fullsized":"2024-04-25T11:39:28.974546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:28.974460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:28.974460+0000","last_clean_scrub_stamp":"2024-04-25T11:39:28.974460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:29.221507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:29.684351+0000","last_change":"2024-04-25T11:39:29.684384+0000","last_active":"2024-04-25T11:39:29.684351+0000","last_peered":"2024-04-25T11:39:29.684351+0000","last_clean":"2024-04-25T11:39:29.684351+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:39:29.684351+0000","last_undegraded":"2024-04-25T11:39:29.684351+0000","last_fullsized":"2024-04-25T11:39:29.684351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:29.684253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:29.684253+0000","last_clean_scrub_stamp":"2024-04-25T11:39:29.684253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:29.029408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.087903+0000","last_change":"2024-04-25T11:39:26.087933+0000","last_active":"2024-04-25T11:39:26.087903+0000","last_peered":"2024-04-25T11:39:26.087903+0000","last_clean":"2024-04-25T11:39:26.087903+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:39:26.087903+0000","last_undegraded":"2024-04-25T11:39:26.087903+0000","last_fullsized":"2024-04-25T11:39:26.087903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:26.087785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:26.087785+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.087785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:57.979990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.038709+0000","last_change":"2024-04-25T11:39:25.038734+0000","last_active":"2024-04-25T11:39:25.038709+0000","last_peered":"2024-04-25T11:39:25.038709+0000","last_clean":"2024-04-25T11:39:25.038709+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:39:25.038709+0000","last_undegraded":"2024-04-25T11:39:25.038709+0000","last_fullsized":"2024-04-25T11:39:25.038709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:25.038604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:25.038604+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.038604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:08.782323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.938383+0000","last_change":"2024-04-25T11:39:17.938412+0000","last_active":"2024-04-25T11:39:17.938383+0000","last_peered":"2024-04-25T11:39:17.938383+0000","last_clean":"2024-04-25T11:39:17.938383+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:39:17.938383+0000","last_undegraded":"2024-04-25T11:39:17.938383+0000","last_fullsized":"2024-04-25T11:39:17.938383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:17.938284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:17.938284+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.938284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:11.056088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.004330+0000","last_change":"2024-04-25T11:39:20.004378+0000","last_active":"2024-04-25T11:39:20.004330+0000","last_peered":"2024-04-25T11:39:20.004330+0000","last_clean":"2024-04-25T11:39:20.004330+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:39:20.004330+0000","last_undegraded":"2024-04-25T11:39:20.004330+0000","last_fullsized":"2024-04-25T11:39:20.004330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:20.004228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:20.004228+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.004228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:55.047057+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.977297+0000","last_change":"2024-04-25T11:39:19.977326+0000","last_active":"2024-04-25T11:39:19.977297+0000","last_peered":"2024-04-25T11:39:19.977297+0000","last_clean":"2024-04-25T11:39:19.977297+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:39:19.977297+0000","last_undegraded":"2024-04-25T11:39:19.977297+0000","last_fullsized":"2024-04-25T11:39:19.977297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:19.977205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:19.977205+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.977205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:00.219313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.044442+0000","last_change":"2024-04-25T11:39:26.044467+0000","last_active":"2024-04-25T11:39:26.044442+0000","last_peered":"2024-04-25T11:39:26.044442+0000","last_clean":"2024-04-25T11:39:26.044442+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:39:26.044442+0000","last_undegraded":"2024-04-25T11:39:26.044442+0000","last_fullsized":"2024-04-25T11:39:26.044442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:26.044355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:26.044355+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.044355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:18.304696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:29.796072+0000","last_change":"2024-04-25T11:39:29.796114+0000","last_active":"2024-04-25T11:39:29.796072+0000","last_peered":"2024-04-25T11:39:29.796072+0000","last_clean":"2024-04-25T11:39:29.796072+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:39:29.796072+0000","last_undegraded":"2024-04-25T11:39:29.796072+0000","last_fullsized":"2024-04-25T11:39:29.796072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:29.795950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:29.795950+0000","last_clean_scrub_stamp":"2024-04-25T11:39:29.795950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:29.141105+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.760439+0000","last_change":"2024-04-25T11:39:23.760466+0000","last_active":"2024-04-25T11:39:23.760439+0000","last_peered":"2024-04-25T11:39:23.760439+0000","last_clean":"2024-04-25T11:39:23.760439+0000","last_became_active":"2024-04-25T11:39:14.575071+0000","last_became_peered":"2024-04-25T11:39:14.575071+0000","last_unstale":"2024-04-25T11:39:23.760439+0000","last_undegraded":"2024-04-25T11:39:23.760439+0000","last_fullsized":"2024-04-25T11:39:23.760439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.760345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.760345+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.760345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.704074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.728743+0000","last_change":"2024-04-25T11:39:20.728768+0000","last_active":"2024-04-25T11:39:20.728743+0000","last_peered":"2024-04-25T11:39:20.728743+0000","last_clean":"2024-04-25T11:39:20.728743+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:39:20.728743+0000","last_undegraded":"2024-04-25T11:39:20.728743+0000","last_fullsized":"2024-04-25T11:39:20.728743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:20.728643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:20.728643+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.728643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:27.926695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:28.649009+0000","last_change":"2024-04-25T11:39:28.649064+0000","last_active":"2024-04-25T11:39:28.649009+0000","last_peered":"2024-04-25T11:39:28.649009+0000","last_clean":"2024-04-25T11:39:28.649009+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:39:28.649009+0000","last_undegraded":"2024-04-25T11:39:28.649009+0000","last_fullsized":"2024-04-25T11:39:28.649009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:28.648913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:28.648913+0000","last_clean_scrub_stamp":"2024-04-25T11:39:28.648913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:19.386900+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.075401+0000","last_change":"2024-04-25T11:39:23.075425+0000","last_active":"2024-04-25T11:39:23.075401+0000","last_peered":"2024-04-25T11:39:23.075401+0000","last_clean":"2024-04-25T11:39:23.075401+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:39:23.075401+0000","last_undegraded":"2024-04-25T11:39:23.075401+0000","last_fullsized":"2024-04-25T11:39:23.075401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.075313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.075313+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.075313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:03.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.946653+0000","last_change":"2024-04-25T11:39:22.946676+0000","last_active":"2024-04-25T11:39:22.946653+0000","last_peered":"2024-04-25T11:39:22.946653+0000","last_clean":"2024-04-25T11:39:22.946653+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:39:22.946653+0000","last_undegraded":"2024-04-25T11:39:22.946653+0000","last_fullsized":"2024-04-25T11:39:22.946653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.946545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.946545+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.946545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:14.869729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.858410+0000","last_change":"2024-04-25T11:39:27.858449+0000","last_active":"2024-04-25T11:39:27.858410+0000","last_peered":"2024-04-25T11:39:27.858410+0000","last_clean":"2024-04-25T11:39:27.858410+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:39:27.858410+0000","last_undegraded":"2024-04-25T11:39:27.858410+0000","last_fullsized":"2024-04-25T11:39:27.858410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.858287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.858287+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.858287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:48.538945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.713470+0000","last_change":"2024-04-25T11:39:24.713498+0000","last_active":"2024-04-25T11:39:24.713470+0000","last_peered":"2024-04-25T11:39:24.713470+0000","last_clean":"2024-04-25T11:39:24.713470+0000","last_became_active":"2024-04-25T11:39:14.942828+0000","last_became_peered":"2024-04-25T11:39:14.942828+0000","last_unstale":"2024-04-25T11:39:24.713470+0000","last_undegraded":"2024-04-25T11:39:24.713470+0000","last_fullsized":"2024-04-25T11:39:24.713470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.713377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.713377+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.713377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:04.955485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.930562+0000","last_change":"2024-04-25T11:39:22.930602+0000","last_active":"2024-04-25T11:39:22.930562+0000","last_peered":"2024-04-25T11:39:22.930562+0000","last_clean":"2024-04-25T11:39:22.930562+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:39:22.930562+0000","last_undegraded":"2024-04-25T11:39:22.930562+0000","last_fullsized":"2024-04-25T11:39:22.930562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.930475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.930475+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.930475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:16.186715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.988787+0000","last_change":"2024-04-25T11:39:22.988812+0000","last_active":"2024-04-25T11:39:22.988787+0000","last_peered":"2024-04-25T11:39:22.988787+0000","last_clean":"2024-04-25T11:39:22.988787+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:39:22.988787+0000","last_undegraded":"2024-04-25T11:39:22.988787+0000","last_fullsized":"2024-04-25T11:39:22.988787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.988690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.988690+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.988690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:26.509366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.136173+0000","last_change":"2024-04-25T11:39:27.136199+0000","last_active":"2024-04-25T11:39:27.136173+0000","last_peered":"2024-04-25T11:39:27.136173+0000","last_clean":"2024-04-25T11:39:27.136173+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:39:27.136173+0000","last_undegraded":"2024-04-25T11:39:27.136173+0000","last_fullsized":"2024-04-25T11:39:27.136173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.136089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.136089+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.136089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:13.625578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.982738+0000","last_change":"2024-04-25T11:39:21.982776+0000","last_active":"2024-04-25T11:39:21.982738+0000","last_peered":"2024-04-25T11:39:21.982738+0000","last_clean":"2024-04-25T11:39:21.982738+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:39:21.982738+0000","last_undegraded":"2024-04-25T11:39:21.982738+0000","last_fullsized":"2024-04-25T11:39:21.982738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.982636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.982636+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.982636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:57.025465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.947073+0000","last_change":"2024-04-25T11:39:23.947095+0000","last_active":"2024-04-25T11:39:23.947073+0000","last_peered":"2024-04-25T11:39:23.947073+0000","last_clean":"2024-04-25T11:39:23.947073+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:39:23.947073+0000","last_undegraded":"2024-04-25T11:39:23.947073+0000","last_fullsized":"2024-04-25T11:39:23.947073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.946992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.946992+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.946992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:16.207333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.987811+0000","last_change":"2024-04-25T11:39:21.987858+0000","last_active":"2024-04-25T11:39:21.987811+0000","last_peered":"2024-04-25T11:39:21.987811+0000","last_clean":"2024-04-25T11:39:21.987811+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:39:21.987811+0000","last_undegraded":"2024-04-25T11:39:21.987811+0000","last_fullsized":"2024-04-25T11:39:21.987811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.987687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.987687+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.987687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:05.731406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.902736+0000","last_change":"2024-04-25T11:39:25.902762+0000","last_active":"2024-04-25T11:39:25.902736+0000","last_peered":"2024-04-25T11:39:25.902736+0000","last_clean":"2024-04-25T11:39:25.902736+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:39:25.902736+0000","last_undegraded":"2024-04-25T11:39:25.902736+0000","last_fullsized":"2024-04-25T11:39:25.902736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:25.902639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:25.902639+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.902639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:01.413098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.716528+0000","last_change":"2024-04-25T11:39:21.716555+0000","last_active":"2024-04-25T11:39:21.716528+0000","last_peered":"2024-04-25T11:39:21.716528+0000","last_clean":"2024-04-25T11:39:21.716528+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:39:21.716528+0000","last_undegraded":"2024-04-25T11:39:21.716528+0000","last_fullsized":"2024-04-25T11:39:21.716528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.716415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.716415+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.716415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:27.861042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:28.158718+0000","last_change":"2024-04-25T11:39:28.158741+0000","last_active":"2024-04-25T11:39:28.158718+0000","last_peered":"2024-04-25T11:39:28.158718+0000","last_clean":"2024-04-25T11:39:28.158718+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:39:28.158718+0000","last_undegraded":"2024-04-25T11:39:28.158718+0000","last_fullsized":"2024-04-25T11:39:28.158718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:28.158622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:28.158622+0000","last_clean_scrub_stamp":"2024-04-25T11:39:28.158622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:40.687977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.979390+0000","last_change":"2024-04-25T11:39:19.979414+0000","last_active":"2024-04-25T11:39:19.979390+0000","last_peered":"2024-04-25T11:39:19.979390+0000","last_clean":"2024-04-25T11:39:19.979390+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:39:19.979390+0000","last_undegraded":"2024-04-25T11:39:19.979390+0000","last_fullsized":"2024-04-25T11:39:19.979390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:19.979304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:19.979304+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.979304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:13.016568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'8","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.013123+0000","last_change":"2024-04-25T11:39:19.013174+0000","last_active":"2024-04-25T11:39:19.013123+0000","last_peered":"2024-04-25T11:39:19.013123+0000","last_clean":"2024-04-25T11:39:19.013123+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:39:19.013123+0000","last_undegraded":"2024-04-25T11:39:19.013123+0000","last_fullsized":"2024-04-25T11:39:19.013123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'8","last_scrub_stamp":"2024-04-25T11:39:19.013023+0000","last_deep_scrub":"22'8","last_deep_scrub_stamp":"2024-04-25T11:39:19.013023+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.013023+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:23.548213+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.954089+0000","last_change":"2024-04-25T11:39:24.954116+0000","last_active":"2024-04-25T11:39:24.954089+0000","last_peered":"2024-04-25T11:39:24.954089+0000","last_clean":"2024-04-25T11:39:24.954089+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:39:24.954089+0000","last_undegraded":"2024-04-25T11:39:24.954089+0000","last_fullsized":"2024-04-25T11:39:24.954089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.953997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.953997+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.953997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:04.561270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'10","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.148047+0000","last_change":"2024-04-25T11:39:18.148081+0000","last_active":"2024-04-25T11:39:18.148047+0000","last_peered":"2024-04-25T11:39:18.148047+0000","last_clean":"2024-04-25T11:39:18.148047+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:39:18.148047+0000","last_undegraded":"2024-04-25T11:39:18.148047+0000","last_fullsized":"2024-04-25T11:39:18.148047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'10","last_scrub_stamp":"2024-04-25T11:39:18.147958+0000","last_deep_scrub":"22'10","last_deep_scrub_stamp":"2024-04-25T11:39:18.147958+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.147958+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:24.903489+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'4","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:16.967040+0000","last_change":"2024-04-25T11:39:16.967065+0000","last_active":"2024-04-25T11:39:16.967040+0000","last_peered":"2024-04-25T11:39:16.967040+0000","last_clean":"2024-04-25T11:39:16.967040+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:39:16.967040+0000","last_undegraded":"2024-04-25T11:39:16.967040+0000","last_fullsized":"2024-04-25T11:39:16.967040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T11:39:16.966979+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T11:39:16.966979+0000","last_clean_scrub_stamp":"2024-04-25T11:39:16.966979+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:17.068337+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.933182+0000","last_change":"2024-04-25T11:39:27.933210+0000","last_active":"2024-04-25T11:39:27.933182+0000","last_peered":"2024-04-25T11:39:27.933182+0000","last_clean":"2024-04-25T11:39:27.933182+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:39:27.933182+0000","last_undegraded":"2024-04-25T11:39:27.933182+0000","last_fullsized":"2024-04-25T11:39:27.933182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.933086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.933086+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.933086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:03.443546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'4","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.889884+0000","last_change":"2024-04-25T11:39:19.889920+0000","last_active":"2024-04-25T11:39:19.889884+0000","last_peered":"2024-04-25T11:39:19.889884+0000","last_clean":"2024-04-25T11:39:19.889884+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:39:19.889884+0000","last_undegraded":"2024-04-25T11:39:19.889884+0000","last_fullsized":"2024-04-25T11:39:19.889884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T11:39:19.889781+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T11:39:19.889781+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.889781+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:11.812965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.977871+0000","last_change":"2024-04-25T11:39:22.977904+0000","last_active":"2024-04-25T11:39:22.977871+0000","last_peered":"2024-04-25T11:39:22.977871+0000","last_clean":"2024-04-25T11:39:22.977871+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:39:22.977871+0000","last_undegraded":"2024-04-25T11:39:22.977871+0000","last_fullsized":"2024-04-25T11:39:22.977871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.977782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.977782+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.977782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:57.881453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'7","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.030271+0000","last_change":"2024-04-25T11:39:21.030300+0000","last_active":"2024-04-25T11:39:21.030271+0000","last_peered":"2024-04-25T11:39:21.030271+0000","last_clean":"2024-04-25T11:39:21.030271+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:39:21.030271+0000","last_undegraded":"2024-04-25T11:39:21.030271+0000","last_fullsized":"2024-04-25T11:39:21.030271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T11:39:21.030186+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T11:39:21.030186+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.030186+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:21.131544+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.902500+0000","last_change":"2024-04-25T11:39:26.902526+0000","last_active":"2024-04-25T11:39:26.902500+0000","last_peered":"2024-04-25T11:39:26.902500+0000","last_clean":"2024-04-25T11:39:26.902500+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:39:26.902500+0000","last_undegraded":"2024-04-25T11:39:26.902500+0000","last_fullsized":"2024-04-25T11:39:26.902500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:26.902413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:26.902413+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.902413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:27.149460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'5","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.941088+0000","last_change":"2024-04-25T11:39:21.941124+0000","last_active":"2024-04-25T11:39:21.941088+0000","last_peered":"2024-04-25T11:39:21.941088+0000","last_clean":"2024-04-25T11:39:21.941088+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:39:21.941088+0000","last_undegraded":"2024-04-25T11:39:21.941088+0000","last_fullsized":"2024-04-25T11:39:21.941088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5","last_scrub_stamp":"2024-04-25T11:39:21.940974+0000","last_deep_scrub":"22'5","last_deep_scrub_stamp":"2024-04-25T11:39:21.940974+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.940974+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:34.470329+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.018506+0000","last_change":"2024-04-25T11:39:21.018545+0000","last_active":"2024-04-25T11:39:21.018506+0000","last_peered":"2024-04-25T11:39:21.018506+0000","last_clean":"2024-04-25T11:39:21.018506+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:39:21.018506+0000","last_undegraded":"2024-04-25T11:39:21.018506+0000","last_fullsized":"2024-04-25T11:39:21.018506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.018409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.018409+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.018409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:30.555823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'9","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.067256+0000","last_change":"2024-04-25T11:39:20.067289+0000","last_active":"2024-04-25T11:39:20.067256+0000","last_peered":"2024-04-25T11:39:20.067256+0000","last_clean":"2024-04-25T11:39:20.067256+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:39:20.067256+0000","last_undegraded":"2024-04-25T11:39:20.067256+0000","last_fullsized":"2024-04-25T11:39:20.067256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T11:39:20.067172+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T11:39:20.067172+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.067172+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:21.741738+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.985560+0000","last_change":"2024-04-25T11:39:20.985585+0000","last_active":"2024-04-25T11:39:20.985560+0000","last_peered":"2024-04-25T11:39:20.985560+0000","last_clean":"2024-04-25T11:39:20.985560+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:39:20.985560+0000","last_undegraded":"2024-04-25T11:39:20.985560+0000","last_fullsized":"2024-04-25T11:39:20.985560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:20.985469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:20.985469+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.985469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:27.741001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'17","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.080682+0000","last_change":"2024-04-25T11:39:20.080721+0000","last_active":"2024-04-25T11:39:20.080682+0000","last_peered":"2024-04-25T11:39:20.080682+0000","last_clean":"2024-04-25T11:39:20.080682+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:39:20.080682+0000","last_undegraded":"2024-04-25T11:39:20.080682+0000","last_fullsized":"2024-04-25T11:39:20.080682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'17","last_scrub_stamp":"2024-04-25T11:39:20.080598+0000","last_deep_scrub":"22'17","last_deep_scrub_stamp":"2024-04-25T11:39:20.080598+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.080598+0000","objects_scrubbed":6,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:26.225225+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.632332+0000","last_change":"2024-04-25T11:39:27.632375+0000","last_active":"2024-04-25T11:39:27.632332+0000","last_peered":"2024-04-25T11:39:27.632332+0000","last_clean":"2024-04-25T11:39:27.632332+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:39:27.632332+0000","last_undegraded":"2024-04-25T11:39:27.632332+0000","last_fullsized":"2024-04-25T11:39:27.632332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.632221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.632221+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.632221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:46.048609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'13","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:14.948141+0000","last_change":"2024-04-25T11:39:14.948195+0000","last_active":"2024-04-25T11:39:14.948141+0000","last_peered":"2024-04-25T11:39:14.948141+0000","last_clean":"2024-04-25T11:39:14.948141+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:39:14.948141+0000","last_undegraded":"2024-04-25T11:39:14.948141+0000","last_fullsized":"2024-04-25T11:39:14.948141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'13","last_scrub_stamp":"2024-04-25T11:39:14.948025+0000","last_deep_scrub":"22'13","last_deep_scrub_stamp":"2024-04-25T11:39:14.948025+0000","last_clean_scrub_stamp":"2024-04-25T11:39:14.948025+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:47.555902+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.952531+0000","last_change":"2024-04-25T11:39:23.952557+0000","last_active":"2024-04-25T11:39:23.952531+0000","last_peered":"2024-04-25T11:39:23.952531+0000","last_clean":"2024-04-25T11:39:23.952531+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:39:23.952531+0000","last_undegraded":"2024-04-25T11:39:23.952531+0000","last_fullsized":"2024-04-25T11:39:23.952531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.952446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.952446+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.952446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:16.212787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'7","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.907650+0000","last_change":"2024-04-25T11:39:18.907684+0000","last_active":"2024-04-25T11:39:18.907650+0000","last_peered":"2024-04-25T11:39:18.907650+0000","last_clean":"2024-04-25T11:39:18.907650+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:39:18.907650+0000","last_undegraded":"2024-04-25T11:39:18.907650+0000","last_fullsized":"2024-04-25T11:39:18.907650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T11:39:18.907564+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T11:39:18.907564+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.907564+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:43.024853+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:14.574379+0000","last_change":"2024-04-25T11:38:25.684128+0000","last_active":"2024-04-25T11:39:14.574379+0000","last_peered":"2024-04-25T11:39:14.574379+0000","last_clean":"2024-04-25T11:39:14.574379+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:39:14.574379+0000","last_undegraded":"2024-04-25T11:39:14.574379+0000","last_fullsized":"2024-04-25T11:39:14.574379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'9","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.102937+0000","last_change":"2024-04-25T11:39:19.102998+0000","last_active":"2024-04-25T11:39:19.102937+0000","last_peered":"2024-04-25T11:39:19.102937+0000","last_clean":"2024-04-25T11:39:19.102937+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:39:19.102937+0000","last_undegraded":"2024-04-25T11:39:19.102937+0000","last_fullsized":"2024-04-25T11:39:19.102937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T11:39:19.102814+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T11:39:19.102814+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.102814+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:26.300866+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:29.114848+0000","last_change":"2024-04-25T11:39:29.114878+0000","last_active":"2024-04-25T11:39:29.114848+0000","last_peered":"2024-04-25T11:39:29.114848+0000","last_clean":"2024-04-25T11:39:29.114848+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:39:29.114848+0000","last_undegraded":"2024-04-25T11:39:29.114848+0000","last_fullsized":"2024-04-25T11:39:29.114848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:29.114745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:29.114745+0000","last_clean_scrub_stamp":"2024-04-25T11:39:29.114745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:22.370985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'25","reported_seq":52,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.804826+0000","last_change":"2024-04-25T11:39:18.804867+0000","last_active":"2024-04-25T11:39:18.804826+0000","last_peered":"2024-04-25T11:39:18.804826+0000","last_clean":"2024-04-25T11:39:18.804826+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:39:18.804826+0000","last_undegraded":"2024-04-25T11:39:18.804826+0000","last_fullsized":"2024-04-25T11:39:18.804826+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'25","last_scrub_stamp":"2024-04-25T11:39:18.804745+0000","last_deep_scrub":"22'25","last_deep_scrub_stamp":"2024-04-25T11:39:18.804745+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.804745+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:11.063457+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:14.574131+0000","last_change":"2024-04-25T11:38:25.682669+0000","last_active":"2024-04-25T11:39:14.574131+0000","last_peered":"2024-04-25T11:39:14.574131+0000","last_clean":"2024-04-25T11:39:14.574131+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:39:14.574131+0000","last_undegraded":"2024-04-25T11:39:14.574131+0000","last_fullsized":"2024-04-25T11:39:14.574131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:38:24.640920+0000","last_clean_scrub_stamp":"2024-04-25T11:38:24.640920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'3","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:16.975752+0000","last_change":"2024-04-25T11:39:16.975782+0000","last_active":"2024-04-25T11:39:16.975752+0000","last_peered":"2024-04-25T11:39:16.975752+0000","last_clean":"2024-04-25T11:39:16.975752+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:39:16.975752+0000","last_undegraded":"2024-04-25T11:39:16.975752+0000","last_fullsized":"2024-04-25T11:39:16.975752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2024-04-25T11:39:16.975657+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2024-04-25T11:39:16.975657+0000","last_clean_scrub_stamp":"2024-04-25T11:39:16.975657+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:17.312794+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:31.715555+0000","last_change":"2024-04-25T11:39:31.715582+0000","last_active":"2024-04-25T11:39:31.715555+0000","last_peered":"2024-04-25T11:39:31.715555+0000","last_clean":"2024-04-25T11:39:31.715555+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:39:31.715555+0000","last_undegraded":"2024-04-25T11:39:31.715555+0000","last_fullsized":"2024-04-25T11:39:31.715555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:31.715459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:31.715459+0000","last_clean_scrub_stamp":"2024-04-25T11:39:31.715459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:06.952171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'6","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.752493+0000","last_change":"2024-04-25T11:39:17.752537+0000","last_active":"2024-04-25T11:39:17.752493+0000","last_peered":"2024-04-25T11:39:17.752493+0000","last_clean":"2024-04-25T11:39:17.752493+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:39:17.752493+0000","last_undegraded":"2024-04-25T11:39:17.752493+0000","last_fullsized":"2024-04-25T11:39:17.752493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6","last_scrub_stamp":"2024-04-25T11:39:17.752385+0000","last_deep_scrub":"22'6","last_deep_scrub_stamp":"2024-04-25T11:39:17.752385+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.752385+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:10.789648+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:32.689067+0000","last_change":"2024-04-25T11:39:32.689093+0000","last_active":"2024-04-25T11:39:32.689067+0000","last_peered":"2024-04-25T11:39:32.689067+0000","last_clean":"2024-04-25T11:39:32.689067+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:39:32.689067+0000","last_undegraded":"2024-04-25T11:39:32.689067+0000","last_fullsized":"2024-04-25T11:39:32.689067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:32.688973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:32.688973+0000","last_clean_scrub_stamp":"2024-04-25T11:39:32.688973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:01.616854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'4","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.033205+0000","last_change":"2024-04-25T11:39:18.033230+0000","last_active":"2024-04-25T11:39:18.033205+0000","last_peered":"2024-04-25T11:39:18.033205+0000","last_clean":"2024-04-25T11:39:18.033205+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:39:18.033205+0000","last_undegraded":"2024-04-25T11:39:18.033205+0000","last_fullsized":"2024-04-25T11:39:18.033205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T11:39:18.033145+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T11:39:18.033145+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.033145+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:15.822794+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.893787+0000","last_change":"2024-04-25T11:39:23.893829+0000","last_active":"2024-04-25T11:39:23.893787+0000","last_peered":"2024-04-25T11:39:23.893787+0000","last_clean":"2024-04-25T11:39:23.893787+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:39:23.893787+0000","last_undegraded":"2024-04-25T11:39:23.893787+0000","last_fullsized":"2024-04-25T11:39:23.893787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.893704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.893704+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.893704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:10.187188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'9","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.758095+0000","last_change":"2024-04-25T11:39:19.758170+0000","last_active":"2024-04-25T11:39:19.758095+0000","last_peered":"2024-04-25T11:39:19.758095+0000","last_clean":"2024-04-25T11:39:19.758095+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:39:19.758095+0000","last_undegraded":"2024-04-25T11:39:19.758095+0000","last_fullsized":"2024-04-25T11:39:19.758095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T11:39:19.757978+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T11:39:19.757978+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.757978+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:52.365855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.619262+0000","last_change":"2024-04-25T11:39:18.619288+0000","last_active":"2024-04-25T11:39:18.619262+0000","last_peered":"2024-04-25T11:39:18.619262+0000","last_clean":"2024-04-25T11:39:18.619262+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:39:18.619262+0000","last_undegraded":"2024-04-25T11:39:18.619262+0000","last_fullsized":"2024-04-25T11:39:18.619262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:18.619164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:18.619164+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.619164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:02.362883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'119","reported_seq":184,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.605564+0000","last_change":"2024-04-25T11:39:20.605604+0000","last_active":"2024-04-25T11:39:20.605564+0000","last_peered":"2024-04-25T11:39:20.605564+0000","last_clean":"2024-04-25T11:39:20.605564+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:39:20.605564+0000","last_undegraded":"2024-04-25T11:39:20.605564+0000","last_fullsized":"2024-04-25T11:39:20.605564+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'119","last_scrub_stamp":"2024-04-25T11:39:20.605477+0000","last_deep_scrub":"22'119","last_deep_scrub_stamp":"2024-04-25T11:39:20.605477+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.605477+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:24.126153+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'10","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.909392+0000","last_change":"2024-04-25T11:39:20.909430+0000","last_active":"2024-04-25T11:39:20.909392+0000","last_peered":"2024-04-25T11:39:20.909392+0000","last_clean":"2024-04-25T11:39:20.909392+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:39:20.909392+0000","last_undegraded":"2024-04-25T11:39:20.909392+0000","last_fullsized":"2024-04-25T11:39:20.909392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'10","last_scrub_stamp":"2024-04-25T11:39:20.909304+0000","last_deep_scrub":"22'10","last_deep_scrub_stamp":"2024-04-25T11:39:20.909304+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.909304+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:07.398793+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.947123+0000","last_change":"2024-04-25T11:39:24.947155+0000","last_active":"2024-04-25T11:39:24.947123+0000","last_peered":"2024-04-25T11:39:24.947123+0000","last_clean":"2024-04-25T11:39:24.947123+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:39:24.947123+0000","last_undegraded":"2024-04-25T11:39:24.947123+0000","last_fullsized":"2024-04-25T11:39:24.947123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.947040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.947040+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.947040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:04.554313+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'5","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.606611+0000","last_change":"2024-04-25T11:39:24.606646+0000","last_active":"2024-04-25T11:39:24.606611+0000","last_peered":"2024-04-25T11:39:24.606611+0000","last_clean":"2024-04-25T11:39:24.606611+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:39:24.606611+0000","last_undegraded":"2024-04-25T11:39:24.606611+0000","last_fullsized":"2024-04-25T11:39:24.606611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5","last_scrub_stamp":"2024-04-25T11:39:24.606523+0000","last_deep_scrub":"22'5","last_deep_scrub_stamp":"2024-04-25T11:39:24.606523+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.606523+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:27.509086+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:16.965777+0000","last_change":"2024-04-25T11:39:16.965816+0000","last_active":"2024-04-25T11:39:16.965777+0000","last_peered":"2024-04-25T11:39:16.965777+0000","last_clean":"2024-04-25T11:39:16.965777+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:39:16.965777+0000","last_undegraded":"2024-04-25T11:39:16.965777+0000","last_fullsized":"2024-04-25T11:39:16.965777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:16.965687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:16.965687+0000","last_clean_scrub_stamp":"2024-04-25T11:39:16.965687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:35.135027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'7","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:16.733625+0000","last_change":"2024-04-25T11:39:16.733652+0000","last_active":"2024-04-25T11:39:16.733625+0000","last_peered":"2024-04-25T11:39:16.733625+0000","last_clean":"2024-04-25T11:39:16.733625+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:39:16.733625+0000","last_undegraded":"2024-04-25T11:39:16.733625+0000","last_fullsized":"2024-04-25T11:39:16.733625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T11:39:16.733554+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T11:39:16.733554+0000","last_clean_scrub_stamp":"2024-04-25T11:39:16.733554+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:17.729453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.017550+0000","last_change":"2024-04-25T11:39:24.017572+0000","last_active":"2024-04-25T11:39:24.017550+0000","last_peered":"2024-04-25T11:39:24.017550+0000","last_clean":"2024-04-25T11:39:24.017550+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:39:24.017550+0000","last_undegraded":"2024-04-25T11:39:24.017550+0000","last_fullsized":"2024-04-25T11:39:24.017550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.017466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.017466+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.017466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:48.134755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.918845+0000","last_change":"2024-04-25T11:39:17.918908+0000","last_active":"2024-04-25T11:39:17.918845+0000","last_peered":"2024-04-25T11:39:17.918845+0000","last_clean":"2024-04-25T11:39:17.918845+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:39:17.918845+0000","last_undegraded":"2024-04-25T11:39:17.918845+0000","last_fullsized":"2024-04-25T11:39:17.918845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'5","last_scrub_stamp":"2024-04-25T11:39:17.918726+0000","last_deep_scrub":"20'5","last_deep_scrub_stamp":"2024-04-25T11:39:17.918726+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.918726+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:52.822397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.940077+0000","last_change":"2024-04-25T11:39:21.940117+0000","last_active":"2024-04-25T11:39:21.940077+0000","last_peered":"2024-04-25T11:39:21.940077+0000","last_clean":"2024-04-25T11:39:21.940077+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:39:21.940077+0000","last_undegraded":"2024-04-25T11:39:21.940077+0000","last_fullsized":"2024-04-25T11:39:21.940077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.939941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.939941+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.939941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.137993+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":28,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:15.967043+0000","last_change":"2024-04-25T11:39:15.967092+0000","last_active":"2024-04-25T11:39:15.967043+0000","last_peered":"2024-04-25T11:39:15.967043+0000","last_clean":"2024-04-25T11:39:15.967043+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:39:15.967043+0000","last_undegraded":"2024-04-25T11:39:15.967043+0000","last_fullsized":"2024-04-25T11:39:15.967043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-04-25T11:39:15.966956+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-04-25T11:39:15.966956+0000","last_clean_scrub_stamp":"2024-04-25T11:39:15.966956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:08.225668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.725955+0000","last_change":"2024-04-25T11:39:22.725981+0000","last_active":"2024-04-25T11:39:22.725955+0000","last_peered":"2024-04-25T11:39:22.725955+0000","last_clean":"2024-04-25T11:39:22.725955+0000","last_became_active":"2024-04-25T11:39:14.114913+0000","last_became_peered":"2024-04-25T11:39:14.114913+0000","last_unstale":"2024-04-25T11:39:22.725955+0000","last_undegraded":"2024-04-25T11:39:22.725955+0000","last_fullsized":"2024-04-25T11:39:22.725955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.725863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.725863+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.725863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:22.827221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"22'7","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.609388+0000","last_change":"2024-04-25T11:39:25.609424+0000","last_active":"2024-04-25T11:39:25.609388+0000","last_peered":"2024-04-25T11:39:25.609388+0000","last_clean":"2024-04-25T11:39:25.609388+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:39:25.609388+0000","last_undegraded":"2024-04-25T11:39:25.609388+0000","last_fullsized":"2024-04-25T11:39:25.609388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T11:39:25.609295+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T11:39:25.609295+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.609295+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:40.145347+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.999448+0000","last_change":"2024-04-25T11:39:24.999473+0000","last_active":"2024-04-25T11:39:24.999448+0000","last_peered":"2024-04-25T11:39:24.999448+0000","last_clean":"2024-04-25T11:39:24.999448+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:39:24.999448+0000","last_undegraded":"2024-04-25T11:39:24.999448+0000","last_fullsized":"2024-04-25T11:39:24.999448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.999360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.999360+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.999360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:28.520036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'11","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.068245+0000","last_change":"2024-04-25T11:39:19.068279+0000","last_active":"2024-04-25T11:39:19.068245+0000","last_peered":"2024-04-25T11:39:19.068245+0000","last_clean":"2024-04-25T11:39:19.068245+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:39:19.068245+0000","last_undegraded":"2024-04-25T11:39:19.068245+0000","last_fullsized":"2024-04-25T11:39:19.068245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'11","last_scrub_stamp":"2024-04-25T11:39:19.068161+0000","last_deep_scrub":"22'11","last_deep_scrub_stamp":"2024-04-25T11:39:19.068161+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.068161+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:19.169520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.984280+0000","last_change":"2024-04-25T11:39:25.984305+0000","last_active":"2024-04-25T11:39:25.984280+0000","last_peered":"2024-04-25T11:39:25.984280+0000","last_clean":"2024-04-25T11:39:25.984280+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:39:25.984280+0000","last_undegraded":"2024-04-25T11:39:25.984280+0000","last_fullsized":"2024-04-25T11:39:25.984280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:25.984184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:25.984184+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.984184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:41.271682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'10","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.057331+0000","last_change":"2024-04-25T11:39:22.057365+0000","last_active":"2024-04-25T11:39:22.057331+0000","last_peered":"2024-04-25T11:39:22.057331+0000","last_clean":"2024-04-25T11:39:22.057331+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:39:22.057331+0000","last_undegraded":"2024-04-25T11:39:22.057331+0000","last_fullsized":"2024-04-25T11:39:22.057331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'10","last_scrub_stamp":"2024-04-25T11:39:22.057233+0000","last_deep_scrub":"22'10","last_deep_scrub_stamp":"2024-04-25T11:39:22.057233+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.057233+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:23.731799+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.931002+0000","last_change":"2024-04-25T11:39:17.931022+0000","last_active":"2024-04-25T11:39:17.931002+0000","last_peered":"2024-04-25T11:39:17.931002+0000","last_clean":"2024-04-25T11:39:17.931002+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:39:17.931002+0000","last_undegraded":"2024-04-25T11:39:17.931002+0000","last_fullsized":"2024-04-25T11:39:17.931002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:17.930937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:17.930937+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.930937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:47.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'3","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.622937+0000","last_change":"2024-04-25T11:39:26.622972+0000","last_active":"2024-04-25T11:39:26.622937+0000","last_peered":"2024-04-25T11:39:26.622937+0000","last_clean":"2024-04-25T11:39:26.622937+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:39:26.622937+0000","last_undegraded":"2024-04-25T11:39:26.622937+0000","last_fullsized":"2024-04-25T11:39:26.622937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2024-04-25T11:39:26.622844+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2024-04-25T11:39:26.622844+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.622844+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:59.922932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.954801+0000","last_change":"2024-04-25T11:39:18.954835+0000","last_active":"2024-04-25T11:39:18.954801+0000","last_peered":"2024-04-25T11:39:18.954801+0000","last_clean":"2024-04-25T11:39:18.954801+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:39:18.954801+0000","last_undegraded":"2024-04-25T11:39:18.954801+0000","last_fullsized":"2024-04-25T11:39:18.954801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:18.954717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:18.954717+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.954717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:04.008478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'10","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.027729+0000","last_change":"2024-04-25T11:39:21.027774+0000","last_active":"2024-04-25T11:39:21.027729+0000","last_peered":"2024-04-25T11:39:21.027729+0000","last_clean":"2024-04-25T11:39:21.027729+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:39:21.027729+0000","last_undegraded":"2024-04-25T11:39:21.027729+0000","last_fullsized":"2024-04-25T11:39:21.027729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'10","last_scrub_stamp":"2024-04-25T11:39:21.027636+0000","last_deep_scrub":"22'10","last_deep_scrub_stamp":"2024-04-25T11:39:21.027636+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.027636+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:25.562826+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.013764+0000","last_change":"2024-04-25T11:39:22.013788+0000","last_active":"2024-04-25T11:39:22.013764+0000","last_peered":"2024-04-25T11:39:22.013764+0000","last_clean":"2024-04-25T11:39:22.013764+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:39:22.013764+0000","last_undegraded":"2024-04-25T11:39:22.013764+0000","last_fullsized":"2024-04-25T11:39:22.013764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.013664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.013664+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.013664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:46.130953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'4","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.012071+0000","last_change":"2024-04-25T11:39:17.012096+0000","last_active":"2024-04-25T11:39:17.012071+0000","last_peered":"2024-04-25T11:39:17.012071+0000","last_clean":"2024-04-25T11:39:17.012071+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:39:17.012071+0000","last_undegraded":"2024-04-25T11:39:17.012071+0000","last_fullsized":"2024-04-25T11:39:17.012071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T11:39:17.012002+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T11:39:17.012002+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.012002+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:49.619879+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.967162+0000","last_change":"2024-04-25T11:39:26.967188+0000","last_active":"2024-04-25T11:39:26.967162+0000","last_peered":"2024-04-25T11:39:26.967162+0000","last_clean":"2024-04-25T11:39:26.967162+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:39:26.967162+0000","last_undegraded":"2024-04-25T11:39:26.967162+0000","last_fullsized":"2024-04-25T11:39:26.967162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:26.967076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:26.967076+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.967076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.513286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'6","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.584634+0000","last_change":"2024-04-25T11:39:21.584667+0000","last_active":"2024-04-25T11:39:21.584634+0000","last_peered":"2024-04-25T11:39:21.584634+0000","last_clean":"2024-04-25T11:39:21.584634+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:39:21.584634+0000","last_undegraded":"2024-04-25T11:39:21.584634+0000","last_fullsized":"2024-04-25T11:39:21.584634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6","last_scrub_stamp":"2024-04-25T11:39:21.584550+0000","last_deep_scrub":"22'6","last_deep_scrub_stamp":"2024-04-25T11:39:21.584550+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.584550+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:34.113905+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.078553+0000","last_change":"2024-04-25T11:39:24.078579+0000","last_active":"2024-04-25T11:39:24.078553+0000","last_peered":"2024-04-25T11:39:24.078553+0000","last_clean":"2024-04-25T11:39:24.078553+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:39:24.078553+0000","last_undegraded":"2024-04-25T11:39:24.078553+0000","last_fullsized":"2024-04-25T11:39:24.078553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.078465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.078465+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.078465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.615880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:30.755577+0000","last_change":"2024-04-25T11:39:30.755602+0000","last_active":"2024-04-25T11:39:30.755577+0000","last_peered":"2024-04-25T11:39:30.755577+0000","last_clean":"2024-04-25T11:39:30.755577+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:39:30.755577+0000","last_undegraded":"2024-04-25T11:39:30.755577+0000","last_fullsized":"2024-04-25T11:39:30.755577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:30.755487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:30.755487+0000","last_clean_scrub_stamp":"2024-04-25T11:39:30.755487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:05.143451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'8","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.576560+0000","last_change":"2024-04-25T11:39:23.576596+0000","last_active":"2024-04-25T11:39:23.576560+0000","last_peered":"2024-04-25T11:39:23.576560+0000","last_clean":"2024-04-25T11:39:23.576560+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:39:23.576560+0000","last_undegraded":"2024-04-25T11:39:23.576560+0000","last_fullsized":"2024-04-25T11:39:23.576560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'8","last_scrub_stamp":"2024-04-25T11:39:23.576477+0000","last_deep_scrub":"22'8","last_deep_scrub_stamp":"2024-04-25T11:39:23.576477+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.576477+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:38.863975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:28.833323+0000","last_change":"2024-04-25T11:39:28.833348+0000","last_active":"2024-04-25T11:39:28.833323+0000","last_peered":"2024-04-25T11:39:28.833323+0000","last_clean":"2024-04-25T11:39:28.833323+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:39:28.833323+0000","last_undegraded":"2024-04-25T11:39:28.833323+0000","last_fullsized":"2024-04-25T11:39:28.833323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:28.833225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:28.833225+0000","last_clean_scrub_stamp":"2024-04-25T11:39:28.833225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:19.571213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'16","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.610400+0000","last_change":"2024-04-25T11:39:22.610438+0000","last_active":"2024-04-25T11:39:22.610400+0000","last_peered":"2024-04-25T11:39:22.610400+0000","last_clean":"2024-04-25T11:39:22.610400+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:39:22.610400+0000","last_undegraded":"2024-04-25T11:39:22.610400+0000","last_fullsized":"2024-04-25T11:39:22.610400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'16","last_scrub_stamp":"2024-04-25T11:39:22.610318+0000","last_deep_scrub":"22'16","last_deep_scrub_stamp":"2024-04-25T11:39:22.610318+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.610318+0000","objects_scrubbed":7,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:15.866558+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.974992+0000","last_change":"2024-04-25T11:39:25.975058+0000","last_active":"2024-04-25T11:39:25.974992+0000","last_peered":"2024-04-25T11:39:25.974992+0000","last_clean":"2024-04-25T11:39:25.974992+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:39:25.974992+0000","last_undegraded":"2024-04-25T11:39:25.974992+0000","last_fullsized":"2024-04-25T11:39:25.974992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:25.974874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:25.974874+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.974874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:23.557971+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.014789+0000","last_change":"2024-04-25T11:39:18.014815+0000","last_active":"2024-04-25T11:39:18.014789+0000","last_peered":"2024-04-25T11:39:18.014789+0000","last_clean":"2024-04-25T11:39:18.014789+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:39:18.014789+0000","last_undegraded":"2024-04-25T11:39:18.014789+0000","last_fullsized":"2024-04-25T11:39:18.014789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'2","last_scrub_stamp":"2024-04-25T11:39:18.014675+0000","last_deep_scrub":"20'2","last_deep_scrub_stamp":"2024-04-25T11:39:18.014675+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.014675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:19.689241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.024589+0000","last_change":"2024-04-25T11:39:21.024616+0000","last_active":"2024-04-25T11:39:21.024589+0000","last_peered":"2024-04-25T11:39:21.024589+0000","last_clean":"2024-04-25T11:39:21.024589+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:39:21.024589+0000","last_undegraded":"2024-04-25T11:39:21.024589+0000","last_fullsized":"2024-04-25T11:39:21.024589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.024497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.024497+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.024497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:55.928168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'17","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:15.948303+0000","last_change":"2024-04-25T11:39:15.948334+0000","last_active":"2024-04-25T11:39:15.948303+0000","last_peered":"2024-04-25T11:39:15.948303+0000","last_clean":"2024-04-25T11:39:15.948303+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:39:15.948303+0000","last_undegraded":"2024-04-25T11:39:15.948303+0000","last_fullsized":"2024-04-25T11:39:15.948303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'17","last_scrub_stamp":"2024-04-25T11:39:15.948234+0000","last_deep_scrub":"22'17","last_deep_scrub_stamp":"2024-04-25T11:39:15.948234+0000","last_clean_scrub_stamp":"2024-04-25T11:39:15.948234+0000","objects_scrubbed":6,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:23.146286+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11177,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":218,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":22354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":331776,"data_stored":676166,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":87497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":87630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":85028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":87531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":85160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":84648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":323584,"data_stored":674358,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:39:38.679 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:38.680 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:38.680 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T11:38:24.640920+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=38, tm_sec=24, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=39, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:39:38.681 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T11:39:57.836 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:39:57.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:39:58.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:39:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.807+0000 7f08e194a640 1 Processor -- start 2024-04-25T11:39:58.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.807+0000 7f08e194a640 1 -- start start 2024-04-25T11:39:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.808+0000 7f08e194a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f08dc156130 0x7f08dc0c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.808+0000 7f08e194a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08dc15c0f0 0x7f08dc0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.808+0000 7f08e194a640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08dc0c9640 con 0x7f08dc06fee0 2024-04-25T11:39:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.808+0000 7f08e194a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f08dc0ca2e0 con 0x7f08dc15c0f0 2024-04-25T11:39:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.808+0000 7f08e194a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f08dc0ca5c0 con 0x7f08dc156130 2024-04-25T11:39:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.808+0000 7f08da7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08dc15c0f0 0x7f08dc0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.808+0000 7f08da7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08dc15c0f0 0x7f08dc0c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46762/0 (socket says 172.21.15.40:46762) 2024-04-25T11:39:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.808+0000 7f08da7fc640 1 -- 172.21.15.40:0/274053167 learned_addr learned my addr 172.21.15.40:0/274053167 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.809+0000 7f08daffd640 1 --2- 172.21.15.40:0/274053167 >> v2:172.21.15.62:3300/0 conn(0x7f08dc156130 0x7f08dc0c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.809+0000 7f08da7fc640 1 -- 172.21.15.40:0/274053167 >> v2:172.21.15.62:3300/0 conn(0x7f08dc156130 msgr2=0x7f08dc0c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.809+0000 7f08da7fc640 1 --2- 172.21.15.40:0/274053167 >> v2:172.21.15.62:3300/0 conn(0x7f08dc156130 0x7f08dc0c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.809+0000 7f08da7fc640 1 -- 172.21.15.40:0/274053167 >> v1:172.21.15.62:6789/0 conn(0x7f08dc06fee0 legacy=0x7f08dc0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.809+0000 7f08da7fc640 1 -- 172.21.15.40:0/274053167 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc0ca8a0 con 0x7f08dc15c0f0 2024-04-25T11:39:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.809+0000 7f08daffd640 1 --2- 172.21.15.40:0/274053167 >> v2:172.21.15.62:3300/0 conn(0x7f08dc156130 0x7f08dc0c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.809+0000 7f08da7fc640 1 --2- 172.21.15.40:0/274053167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc15c0f0 0x7f08dc0c8d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0c8a10 tx=0x7f08d002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fc9e530a0beee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08d9ffb640 1 -- 172.21.15.40:0/274053167 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0037070 con 0x7f08dc15c0f0 2024-04-25T11:39:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08d9ffb640 1 -- 172.21.15.40:0/274053167 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d002ae50 con 0x7f08dc15c0f0 2024-04-25T11:39:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08d9ffb640 1 -- 172.21.15.40:0/274053167 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0032860 con 0x7f08dc15c0f0 2024-04-25T11:39:58.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08e194a640 1 -- 172.21.15.40:0/274053167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc15c0f0 msgr2=0x7f08dc0c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:58.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08e194a640 1 --2- 172.21.15.40:0/274053167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc15c0f0 0x7f08dc0c8d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0c8a10 tx=0x7f08d002a300 comp rx=0 tx=0).stop 2024-04-25T11:39:58.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08e194a640 1 -- 172.21.15.40:0/274053167 shutdown_connections 2024-04-25T11:39:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08e194a640 1 --2- 172.21.15.40:0/274053167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc15c0f0 0x7f08dc0c8d50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08e194a640 1 --2- 172.21.15.40:0/274053167 >> v2:172.21.15.62:3300/0 conn(0x7f08dc156130 0x7f08dc0c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08e194a640 1 -- 172.21.15.40:0/274053167 >> 172.21.15.40:0/274053167 conn(0x7f08dc06bd90 msgr2=0x7f08dc0676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08e194a640 1 -- 172.21.15.40:0/274053167 shutdown_connections 2024-04-25T11:39:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.810+0000 7f08e194a640 1 -- 172.21.15.40:0/274053167 wait complete. 2024-04-25T11:39:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.812+0000 7f08e194a640 1 Processor -- start 2024-04-25T11:39:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.812+0000 7f08e194a640 1 -- start start 2024-04-25T11:39:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.812+0000 7f08e194a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc06fee0 0x7f08dc0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08e194a640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08dc156130 0x7f08dc1e5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08e194a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08dc15c0f0 0x7f08dc1f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08e194a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc0c9290 con 0x7f08dc06fee0 2024-04-25T11:39:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08e194a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc0ce640 con 0x7f08dc15c0f0 2024-04-25T11:39:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08e194a640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f08dc156510 con 0x7f08dc156130 2024-04-25T11:39:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc06fee0 0x7f08dc0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc06fee0 0x7f08dc0cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46764/0 (socket says 172.21.15.40:46764) 2024-04-25T11:39:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08daffd640 1 -- 172.21.15.40:0/3756284668 learned_addr learned my addr 172.21.15.40:0/3756284668 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08db7fe640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08dc15c0f0 0x7f08dc1f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08da7fc640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08dc156130 0x7f08dc1e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08daffd640 1 -- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08dc156130 msgr2=0x7f08dc1e5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08daffd640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08dc156130 0x7f08dc1e5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08daffd640 1 -- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08dc15c0f0 msgr2=0x7f08dc1f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08daffd640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08dc15c0f0 0x7f08dc1f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08daffd640 1 -- 172.21.15.40:0/3756284668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc0cc690 con 0x7f08dc06fee0 2024-04-25T11:39:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08daffd640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc06fee0 0x7f08dc0cc150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f08d400e810 tx=0x7f08d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08c3fff640 1 -- 172.21.15.40:0/3756284668 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d400f040 con 0x7f08dc06fee0 2024-04-25T11:39:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08c3fff640 1 -- 172.21.15.40:0/3756284668 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d4004030 con 0x7f08dc06fee0 2024-04-25T11:39:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc1518b0 con 0x7f08dc06fee0 2024-04-25T11:39:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08c3fff640 1 -- 172.21.15.40:0/3756284668 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4004400 con 0x7f08dc06fee0 2024-04-25T11:39:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.813+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc0aeb00 con 0x7f08dc06fee0 2024-04-25T11:39:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.815+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08dc068dc0 con 0x7f08dc06fee0 2024-04-25T11:39:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.815+0000 7f08c3fff640 1 -- 172.21.15.40:0/3756284668 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08d400c6d0 con 0x7f08dc06fee0 2024-04-25T11:39:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.816+0000 7f08c3fff640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f08c806c8f0 0x7f08c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.816+0000 7f08c3fff640 1 -- 172.21.15.40:0/3756284668 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f08d408b900 con 0x7f08dc06fee0 2024-04-25T11:39:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.816+0000 7f08da7fc640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f08c806c8f0 0x7f08c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.816+0000 7f08da7fc640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f08c806c8f0 0x7f08c806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f08d0034000 tx=0x7f08d002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:58.820+0000 7f08c3fff640 1 -- 172.21.15.40:0/3756284668 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08dc068dc0 con 0x7f08dc06fee0 2024-04-25T11:39:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.003+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 --> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08dc069200 con 0x7f08c806c8f0 2024-04-25T11:39:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.010+0000 7f08c3fff640 1 -- 172.21.15.40:0/3756284668 <== mgr.4107 v2:172.21.15.62:6832/1406383768 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252137 (secure 0 0 0) 0x7f08d007f900 con 0x7f08c806c8f0 2024-04-25T11:39:59.000 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-25T11:39:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f08c806c8f0 msgr2=0x7f08c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f08c806c8f0 0x7f08c806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f08d0034000 tx=0x7f08d002e000 comp rx=0 tx=0).stop 2024-04-25T11:39:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc06fee0 msgr2=0x7f08dc0cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc06fee0 0x7f08dc0cc150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f08d400e810 tx=0x7f08d400ecd0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 shutdown_connections 2024-04-25T11:39:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f08c806c8f0 0x7f08c806edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08dc15c0f0 0x7f08dc1f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08dc156130 0x7f08dc1e5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 --2- 172.21.15.40:0/3756284668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08dc06fee0 0x7f08dc0cc150 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 >> 172.21.15.40:0/3756284668 conn(0x7f08dc06bd90 msgr2=0x7f08dc0cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 shutdown_connections 2024-04-25T11:39:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.013+0000 7f08e194a640 1 -- 172.21.15.40:0/3756284668 wait complete. 2024-04-25T11:39:59.011 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-25T11:39:58.972227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601545,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":377,"num_write_kb":2491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218304,"kb_used_data":2176,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756416,"statfs":{"total":773094113280,"available":772870569984,"internally_reserved":0,"allocated":2228224,"data_stored":1868018,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003094"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.614289+0000","last_change":"2024-04-25T11:39:19.614312+0000","last_active":"2024-04-25T11:39:19.614289+0000","last_peered":"2024-04-25T11:39:19.614289+0000","last_clean":"2024-04-25T11:39:19.614289+0000","last_became_active":"2024-04-25T11:38:25.677265+0000","last_became_peered":"2024-04-25T11:38:25.677265+0000","last_unstale":"2024-04-25T11:39:19.614289+0000","last_undegraded":"2024-04-25T11:39:19.614289+0000","last_fullsized":"2024-04-25T11:39:19.614289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:19.614191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:19.614191+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.614191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:51.506396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.034115+0000","last_change":"2024-04-25T11:39:27.034141+0000","last_active":"2024-04-25T11:39:27.034115+0000","last_peered":"2024-04-25T11:39:27.034115+0000","last_clean":"2024-04-25T11:39:27.034115+0000","last_became_active":"2024-04-25T11:38:25.677141+0000","last_became_peered":"2024-04-25T11:38:25.677141+0000","last_unstale":"2024-04-25T11:39:27.034115+0000","last_undegraded":"2024-04-25T11:39:27.034115+0000","last_fullsized":"2024-04-25T11:39:27.034115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.034025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.034025+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.034025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:20.290265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.936941+0000","last_change":"2024-04-25T11:39:18.936967+0000","last_active":"2024-04-25T11:39:18.936941+0000","last_peered":"2024-04-25T11:39:18.936941+0000","last_clean":"2024-04-25T11:39:18.936941+0000","last_became_active":"2024-04-25T11:38:25.676169+0000","last_became_peered":"2024-04-25T11:38:25.676169+0000","last_unstale":"2024-04-25T11:39:18.936941+0000","last_undegraded":"2024-04-25T11:39:18.936941+0000","last_fullsized":"2024-04-25T11:39:18.936941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:18.936865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:18.936865+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.936865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:44.880593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.922517+0000","last_change":"2024-04-25T11:39:24.922539+0000","last_active":"2024-04-25T11:39:24.922517+0000","last_peered":"2024-04-25T11:39:24.922517+0000","last_clean":"2024-04-25T11:39:24.922517+0000","last_became_active":"2024-04-25T11:38:25.676247+0000","last_became_peered":"2024-04-25T11:38:25.676247+0000","last_unstale":"2024-04-25T11:39:24.922517+0000","last_undegraded":"2024-04-25T11:39:24.922517+0000","last_fullsized":"2024-04-25T11:39:24.922517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.922439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.922439+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.922439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:32.468649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:30.702804+0000","last_change":"2024-04-25T11:39:30.702842+0000","last_active":"2024-04-25T11:39:30.702804+0000","last_peered":"2024-04-25T11:39:30.702804+0000","last_clean":"2024-04-25T11:39:30.702804+0000","last_became_active":"2024-04-25T11:38:25.677689+0000","last_became_peered":"2024-04-25T11:38:25.677689+0000","last_unstale":"2024-04-25T11:39:30.702804+0000","last_undegraded":"2024-04-25T11:39:30.702804+0000","last_fullsized":"2024-04-25T11:39:30.702804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:30.702702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:30.702702+0000","last_clean_scrub_stamp":"2024-04-25T11:39:30.702702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:05.090666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:31.745951+0000","last_change":"2024-04-25T11:39:31.745976+0000","last_active":"2024-04-25T11:39:31.745951+0000","last_peered":"2024-04-25T11:39:31.745951+0000","last_clean":"2024-04-25T11:39:31.745951+0000","last_became_active":"2024-04-25T11:38:25.683611+0000","last_became_peered":"2024-04-25T11:38:25.683611+0000","last_unstale":"2024-04-25T11:39:31.745951+0000","last_undegraded":"2024-04-25T11:39:31.745951+0000","last_fullsized":"2024-04-25T11:39:31.745951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:31.745860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:31.745860+0000","last_clean_scrub_stamp":"2024-04-25T11:39:31.745860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:06.982572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:34.619496+0000","last_change":"2024-04-25T11:39:34.619519+0000","last_active":"2024-04-25T11:39:34.619496+0000","last_peered":"2024-04-25T11:39:34.619496+0000","last_clean":"2024-04-25T11:39:34.619496+0000","last_became_active":"2024-04-25T11:38:25.682361+0000","last_became_peered":"2024-04-25T11:38:25.682361+0000","last_unstale":"2024-04-25T11:39:34.619496+0000","last_undegraded":"2024-04-25T11:39:34.619496+0000","last_fullsized":"2024-04-25T11:39:34.619496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:34.619411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:34.619411+0000","last_clean_scrub_stamp":"2024-04-25T11:39:34.619411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:28.237796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:28.974546+0000","last_change":"2024-04-25T11:39:28.974567+0000","last_active":"2024-04-25T11:39:28.974546+0000","last_peered":"2024-04-25T11:39:28.974546+0000","last_clean":"2024-04-25T11:39:28.974546+0000","last_became_active":"2024-04-25T11:39:14.979698+0000","last_became_peered":"2024-04-25T11:39:14.979698+0000","last_unstale":"2024-04-25T11:39:28.974546+0000","last_undegraded":"2024-04-25T11:39:28.974546+0000","last_fullsized":"2024-04-25T11:39:28.974546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:28.974460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:28.974460+0000","last_clean_scrub_stamp":"2024-04-25T11:39:28.974460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:29.221507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:29.684351+0000","last_change":"2024-04-25T11:39:29.684384+0000","last_active":"2024-04-25T11:39:29.684351+0000","last_peered":"2024-04-25T11:39:29.684351+0000","last_clean":"2024-04-25T11:39:29.684351+0000","last_became_active":"2024-04-25T11:38:25.677665+0000","last_became_peered":"2024-04-25T11:38:25.677665+0000","last_unstale":"2024-04-25T11:39:29.684351+0000","last_undegraded":"2024-04-25T11:39:29.684351+0000","last_fullsized":"2024-04-25T11:39:29.684351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:29.684253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:29.684253+0000","last_clean_scrub_stamp":"2024-04-25T11:39:29.684253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:29.029408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.087903+0000","last_change":"2024-04-25T11:39:26.087933+0000","last_active":"2024-04-25T11:39:26.087903+0000","last_peered":"2024-04-25T11:39:26.087903+0000","last_clean":"2024-04-25T11:39:26.087903+0000","last_became_active":"2024-04-25T11:38:25.675290+0000","last_became_peered":"2024-04-25T11:38:25.675290+0000","last_unstale":"2024-04-25T11:39:26.087903+0000","last_undegraded":"2024-04-25T11:39:26.087903+0000","last_fullsized":"2024-04-25T11:39:26.087903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:26.087785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:26.087785+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.087785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:57.979990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.038709+0000","last_change":"2024-04-25T11:39:25.038734+0000","last_active":"2024-04-25T11:39:25.038709+0000","last_peered":"2024-04-25T11:39:25.038709+0000","last_clean":"2024-04-25T11:39:25.038709+0000","last_became_active":"2024-04-25T11:38:25.670815+0000","last_became_peered":"2024-04-25T11:38:25.670815+0000","last_unstale":"2024-04-25T11:39:25.038709+0000","last_undegraded":"2024-04-25T11:39:25.038709+0000","last_fullsized":"2024-04-25T11:39:25.038709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:25.038604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:25.038604+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.038604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:08.782323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.938383+0000","last_change":"2024-04-25T11:39:17.938412+0000","last_active":"2024-04-25T11:39:17.938383+0000","last_peered":"2024-04-25T11:39:17.938383+0000","last_clean":"2024-04-25T11:39:17.938383+0000","last_became_active":"2024-04-25T11:38:25.665322+0000","last_became_peered":"2024-04-25T11:38:25.665322+0000","last_unstale":"2024-04-25T11:39:17.938383+0000","last_undegraded":"2024-04-25T11:39:17.938383+0000","last_fullsized":"2024-04-25T11:39:17.938383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:17.938284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:17.938284+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.938284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:11.056088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.004330+0000","last_change":"2024-04-25T11:39:20.004378+0000","last_active":"2024-04-25T11:39:20.004330+0000","last_peered":"2024-04-25T11:39:20.004330+0000","last_clean":"2024-04-25T11:39:20.004330+0000","last_became_active":"2024-04-25T11:38:25.666426+0000","last_became_peered":"2024-04-25T11:38:25.666426+0000","last_unstale":"2024-04-25T11:39:20.004330+0000","last_undegraded":"2024-04-25T11:39:20.004330+0000","last_fullsized":"2024-04-25T11:39:20.004330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:20.004228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:20.004228+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.004228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:55.047057+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.977297+0000","last_change":"2024-04-25T11:39:19.977326+0000","last_active":"2024-04-25T11:39:19.977297+0000","last_peered":"2024-04-25T11:39:19.977297+0000","last_clean":"2024-04-25T11:39:19.977297+0000","last_became_active":"2024-04-25T11:38:25.677131+0000","last_became_peered":"2024-04-25T11:38:25.677131+0000","last_unstale":"2024-04-25T11:39:19.977297+0000","last_undegraded":"2024-04-25T11:39:19.977297+0000","last_fullsized":"2024-04-25T11:39:19.977297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:19.977205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:19.977205+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.977205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:00.219313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.044442+0000","last_change":"2024-04-25T11:39:26.044467+0000","last_active":"2024-04-25T11:39:26.044442+0000","last_peered":"2024-04-25T11:39:26.044442+0000","last_clean":"2024-04-25T11:39:26.044442+0000","last_became_active":"2024-04-25T11:38:25.671800+0000","last_became_peered":"2024-04-25T11:38:25.671800+0000","last_unstale":"2024-04-25T11:39:26.044442+0000","last_undegraded":"2024-04-25T11:39:26.044442+0000","last_fullsized":"2024-04-25T11:39:26.044442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:26.044355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:26.044355+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.044355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:18.304696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:29.796072+0000","last_change":"2024-04-25T11:39:29.796114+0000","last_active":"2024-04-25T11:39:29.796072+0000","last_peered":"2024-04-25T11:39:29.796072+0000","last_clean":"2024-04-25T11:39:29.796072+0000","last_became_active":"2024-04-25T11:38:25.682244+0000","last_became_peered":"2024-04-25T11:38:25.682244+0000","last_unstale":"2024-04-25T11:39:29.796072+0000","last_undegraded":"2024-04-25T11:39:29.796072+0000","last_fullsized":"2024-04-25T11:39:29.796072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:29.795950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:29.795950+0000","last_clean_scrub_stamp":"2024-04-25T11:39:29.795950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:29.141105+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.760439+0000","last_change":"2024-04-25T11:39:23.760466+0000","last_active":"2024-04-25T11:39:23.760439+0000","last_peered":"2024-04-25T11:39:23.760439+0000","last_clean":"2024-04-25T11:39:23.760439+0000","last_became_active":"2024-04-25T11:39:14.575071+0000","last_became_peered":"2024-04-25T11:39:14.575071+0000","last_unstale":"2024-04-25T11:39:23.760439+0000","last_undegraded":"2024-04-25T11:39:23.760439+0000","last_fullsized":"2024-04-25T11:39:23.760439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.760345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.760345+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.760345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.704074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.728743+0000","last_change":"2024-04-25T11:39:20.728768+0000","last_active":"2024-04-25T11:39:20.728743+0000","last_peered":"2024-04-25T11:39:20.728743+0000","last_clean":"2024-04-25T11:39:20.728743+0000","last_became_active":"2024-04-25T11:38:25.669031+0000","last_became_peered":"2024-04-25T11:38:25.669031+0000","last_unstale":"2024-04-25T11:39:20.728743+0000","last_undegraded":"2024-04-25T11:39:20.728743+0000","last_fullsized":"2024-04-25T11:39:20.728743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:20.728643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:20.728643+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.728643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:27.926695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:28.649009+0000","last_change":"2024-04-25T11:39:28.649064+0000","last_active":"2024-04-25T11:39:28.649009+0000","last_peered":"2024-04-25T11:39:28.649009+0000","last_clean":"2024-04-25T11:39:28.649009+0000","last_became_active":"2024-04-25T11:38:25.677761+0000","last_became_peered":"2024-04-25T11:38:25.677761+0000","last_unstale":"2024-04-25T11:39:28.649009+0000","last_undegraded":"2024-04-25T11:39:28.649009+0000","last_fullsized":"2024-04-25T11:39:28.649009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:28.648913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:28.648913+0000","last_clean_scrub_stamp":"2024-04-25T11:39:28.648913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:19.386900+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.075401+0000","last_change":"2024-04-25T11:39:23.075425+0000","last_active":"2024-04-25T11:39:23.075401+0000","last_peered":"2024-04-25T11:39:23.075401+0000","last_clean":"2024-04-25T11:39:23.075401+0000","last_became_active":"2024-04-25T11:38:25.666201+0000","last_became_peered":"2024-04-25T11:38:25.666201+0000","last_unstale":"2024-04-25T11:39:23.075401+0000","last_undegraded":"2024-04-25T11:39:23.075401+0000","last_fullsized":"2024-04-25T11:39:23.075401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.075313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.075313+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.075313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:03.317421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.946653+0000","last_change":"2024-04-25T11:39:22.946676+0000","last_active":"2024-04-25T11:39:22.946653+0000","last_peered":"2024-04-25T11:39:22.946653+0000","last_clean":"2024-04-25T11:39:22.946653+0000","last_became_active":"2024-04-25T11:38:25.679769+0000","last_became_peered":"2024-04-25T11:38:25.679769+0000","last_unstale":"2024-04-25T11:39:22.946653+0000","last_undegraded":"2024-04-25T11:39:22.946653+0000","last_fullsized":"2024-04-25T11:39:22.946653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.946545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.946545+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.946545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:14.869729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.858410+0000","last_change":"2024-04-25T11:39:27.858449+0000","last_active":"2024-04-25T11:39:27.858410+0000","last_peered":"2024-04-25T11:39:27.858410+0000","last_clean":"2024-04-25T11:39:27.858410+0000","last_became_active":"2024-04-25T11:38:25.679307+0000","last_became_peered":"2024-04-25T11:38:25.679307+0000","last_unstale":"2024-04-25T11:39:27.858410+0000","last_undegraded":"2024-04-25T11:39:27.858410+0000","last_fullsized":"2024-04-25T11:39:27.858410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.858287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.858287+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.858287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:48.538945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.713470+0000","last_change":"2024-04-25T11:39:24.713498+0000","last_active":"2024-04-25T11:39:24.713470+0000","last_peered":"2024-04-25T11:39:24.713470+0000","last_clean":"2024-04-25T11:39:24.713470+0000","last_became_active":"2024-04-25T11:39:14.942828+0000","last_became_peered":"2024-04-25T11:39:14.942828+0000","last_unstale":"2024-04-25T11:39:24.713470+0000","last_undegraded":"2024-04-25T11:39:24.713470+0000","last_fullsized":"2024-04-25T11:39:24.713470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.713377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.713377+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.713377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:04.955485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.930562+0000","last_change":"2024-04-25T11:39:22.930602+0000","last_active":"2024-04-25T11:39:22.930562+0000","last_peered":"2024-04-25T11:39:22.930562+0000","last_clean":"2024-04-25T11:39:22.930562+0000","last_became_active":"2024-04-25T11:38:25.673786+0000","last_became_peered":"2024-04-25T11:38:25.673786+0000","last_unstale":"2024-04-25T11:39:22.930562+0000","last_undegraded":"2024-04-25T11:39:22.930562+0000","last_fullsized":"2024-04-25T11:39:22.930562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.930475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.930475+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.930475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:16.186715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.988787+0000","last_change":"2024-04-25T11:39:22.988812+0000","last_active":"2024-04-25T11:39:22.988787+0000","last_peered":"2024-04-25T11:39:22.988787+0000","last_clean":"2024-04-25T11:39:22.988787+0000","last_became_active":"2024-04-25T11:38:25.671953+0000","last_became_peered":"2024-04-25T11:38:25.671953+0000","last_unstale":"2024-04-25T11:39:22.988787+0000","last_undegraded":"2024-04-25T11:39:22.988787+0000","last_fullsized":"2024-04-25T11:39:22.988787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.988690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.988690+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.988690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:26.509366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.136173+0000","last_change":"2024-04-25T11:39:27.136199+0000","last_active":"2024-04-25T11:39:27.136173+0000","last_peered":"2024-04-25T11:39:27.136173+0000","last_clean":"2024-04-25T11:39:27.136173+0000","last_became_active":"2024-04-25T11:38:25.679492+0000","last_became_peered":"2024-04-25T11:38:25.679492+0000","last_unstale":"2024-04-25T11:39:27.136173+0000","last_undegraded":"2024-04-25T11:39:27.136173+0000","last_fullsized":"2024-04-25T11:39:27.136173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.136089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.136089+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.136089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:13.625578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.982738+0000","last_change":"2024-04-25T11:39:21.982776+0000","last_active":"2024-04-25T11:39:21.982738+0000","last_peered":"2024-04-25T11:39:21.982738+0000","last_clean":"2024-04-25T11:39:21.982738+0000","last_became_active":"2024-04-25T11:38:25.665328+0000","last_became_peered":"2024-04-25T11:38:25.665328+0000","last_unstale":"2024-04-25T11:39:21.982738+0000","last_undegraded":"2024-04-25T11:39:21.982738+0000","last_fullsized":"2024-04-25T11:39:21.982738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.982636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.982636+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.982636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:57.025465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.947073+0000","last_change":"2024-04-25T11:39:23.947095+0000","last_active":"2024-04-25T11:39:23.947073+0000","last_peered":"2024-04-25T11:39:23.947073+0000","last_clean":"2024-04-25T11:39:23.947073+0000","last_became_active":"2024-04-25T11:38:25.672623+0000","last_became_peered":"2024-04-25T11:38:25.672623+0000","last_unstale":"2024-04-25T11:39:23.947073+0000","last_undegraded":"2024-04-25T11:39:23.947073+0000","last_fullsized":"2024-04-25T11:39:23.947073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.946992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.946992+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.946992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:16.207333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.987811+0000","last_change":"2024-04-25T11:39:21.987858+0000","last_active":"2024-04-25T11:39:21.987811+0000","last_peered":"2024-04-25T11:39:21.987811+0000","last_clean":"2024-04-25T11:39:21.987811+0000","last_became_active":"2024-04-25T11:38:25.679092+0000","last_became_peered":"2024-04-25T11:38:25.679092+0000","last_unstale":"2024-04-25T11:39:21.987811+0000","last_undegraded":"2024-04-25T11:39:21.987811+0000","last_fullsized":"2024-04-25T11:39:21.987811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.987687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.987687+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.987687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:05.731406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.902736+0000","last_change":"2024-04-25T11:39:25.902762+0000","last_active":"2024-04-25T11:39:25.902736+0000","last_peered":"2024-04-25T11:39:25.902736+0000","last_clean":"2024-04-25T11:39:25.902736+0000","last_became_active":"2024-04-25T11:38:25.678874+0000","last_became_peered":"2024-04-25T11:38:25.678874+0000","last_unstale":"2024-04-25T11:39:25.902736+0000","last_undegraded":"2024-04-25T11:39:25.902736+0000","last_fullsized":"2024-04-25T11:39:25.902736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:25.902639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:25.902639+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.902639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:01.413098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.716528+0000","last_change":"2024-04-25T11:39:21.716555+0000","last_active":"2024-04-25T11:39:21.716528+0000","last_peered":"2024-04-25T11:39:21.716528+0000","last_clean":"2024-04-25T11:39:21.716528+0000","last_became_active":"2024-04-25T11:38:25.669415+0000","last_became_peered":"2024-04-25T11:38:25.669415+0000","last_unstale":"2024-04-25T11:39:21.716528+0000","last_undegraded":"2024-04-25T11:39:21.716528+0000","last_fullsized":"2024-04-25T11:39:21.716528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.716415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.716415+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.716415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:27.861042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:28.158718+0000","last_change":"2024-04-25T11:39:28.158741+0000","last_active":"2024-04-25T11:39:28.158718+0000","last_peered":"2024-04-25T11:39:28.158718+0000","last_clean":"2024-04-25T11:39:28.158718+0000","last_became_active":"2024-04-25T11:38:25.679664+0000","last_became_peered":"2024-04-25T11:38:25.679664+0000","last_unstale":"2024-04-25T11:39:28.158718+0000","last_undegraded":"2024-04-25T11:39:28.158718+0000","last_fullsized":"2024-04-25T11:39:28.158718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:28.158622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:28.158622+0000","last_clean_scrub_stamp":"2024-04-25T11:39:28.158622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:40.687977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.979390+0000","last_change":"2024-04-25T11:39:19.979414+0000","last_active":"2024-04-25T11:39:19.979390+0000","last_peered":"2024-04-25T11:39:19.979390+0000","last_clean":"2024-04-25T11:39:19.979390+0000","last_became_active":"2024-04-25T11:38:25.667775+0000","last_became_peered":"2024-04-25T11:38:25.667775+0000","last_unstale":"2024-04-25T11:39:19.979390+0000","last_undegraded":"2024-04-25T11:39:19.979390+0000","last_fullsized":"2024-04-25T11:39:19.979390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:19.979304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:19.979304+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.979304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:13.016568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'8","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.013123+0000","last_change":"2024-04-25T11:39:19.013174+0000","last_active":"2024-04-25T11:39:19.013123+0000","last_peered":"2024-04-25T11:39:19.013123+0000","last_clean":"2024-04-25T11:39:19.013123+0000","last_became_active":"2024-04-25T11:38:24.665917+0000","last_became_peered":"2024-04-25T11:38:24.665917+0000","last_unstale":"2024-04-25T11:39:19.013123+0000","last_undegraded":"2024-04-25T11:39:19.013123+0000","last_fullsized":"2024-04-25T11:39:19.013123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'8","last_scrub_stamp":"2024-04-25T11:39:19.013023+0000","last_deep_scrub":"22'8","last_deep_scrub_stamp":"2024-04-25T11:39:19.013023+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.013023+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:23.548213+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.954089+0000","last_change":"2024-04-25T11:39:24.954116+0000","last_active":"2024-04-25T11:39:24.954089+0000","last_peered":"2024-04-25T11:39:24.954089+0000","last_clean":"2024-04-25T11:39:24.954089+0000","last_became_active":"2024-04-25T11:38:25.682562+0000","last_became_peered":"2024-04-25T11:38:25.682562+0000","last_unstale":"2024-04-25T11:39:24.954089+0000","last_undegraded":"2024-04-25T11:39:24.954089+0000","last_fullsized":"2024-04-25T11:39:24.954089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.953997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.953997+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.953997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:04.561270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'10","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.148047+0000","last_change":"2024-04-25T11:39:18.148081+0000","last_active":"2024-04-25T11:39:18.148047+0000","last_peered":"2024-04-25T11:39:18.148047+0000","last_clean":"2024-04-25T11:39:18.148047+0000","last_became_active":"2024-04-25T11:38:24.660466+0000","last_became_peered":"2024-04-25T11:38:24.660466+0000","last_unstale":"2024-04-25T11:39:18.148047+0000","last_undegraded":"2024-04-25T11:39:18.148047+0000","last_fullsized":"2024-04-25T11:39:18.148047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'10","last_scrub_stamp":"2024-04-25T11:39:18.147958+0000","last_deep_scrub":"22'10","last_deep_scrub_stamp":"2024-04-25T11:39:18.147958+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.147958+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:24.903489+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'4","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:16.967040+0000","last_change":"2024-04-25T11:39:16.967065+0000","last_active":"2024-04-25T11:39:16.967040+0000","last_peered":"2024-04-25T11:39:16.967040+0000","last_clean":"2024-04-25T11:39:16.967040+0000","last_became_active":"2024-04-25T11:38:24.656433+0000","last_became_peered":"2024-04-25T11:38:24.656433+0000","last_unstale":"2024-04-25T11:39:16.967040+0000","last_undegraded":"2024-04-25T11:39:16.967040+0000","last_fullsized":"2024-04-25T11:39:16.967040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T11:39:16.966979+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T11:39:16.966979+0000","last_clean_scrub_stamp":"2024-04-25T11:39:16.966979+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:17.068337+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.933182+0000","last_change":"2024-04-25T11:39:27.933210+0000","last_active":"2024-04-25T11:39:27.933182+0000","last_peered":"2024-04-25T11:39:27.933182+0000","last_clean":"2024-04-25T11:39:27.933182+0000","last_became_active":"2024-04-25T11:38:25.680684+0000","last_became_peered":"2024-04-25T11:38:25.680684+0000","last_unstale":"2024-04-25T11:39:27.933182+0000","last_undegraded":"2024-04-25T11:39:27.933182+0000","last_fullsized":"2024-04-25T11:39:27.933182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.933086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.933086+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.933086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:03.443546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'4","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.889884+0000","last_change":"2024-04-25T11:39:19.889920+0000","last_active":"2024-04-25T11:39:19.889884+0000","last_peered":"2024-04-25T11:39:19.889884+0000","last_clean":"2024-04-25T11:39:19.889884+0000","last_became_active":"2024-04-25T11:38:24.668572+0000","last_became_peered":"2024-04-25T11:38:24.668572+0000","last_unstale":"2024-04-25T11:39:19.889884+0000","last_undegraded":"2024-04-25T11:39:19.889884+0000","last_fullsized":"2024-04-25T11:39:19.889884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T11:39:19.889781+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T11:39:19.889781+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.889781+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:11.812965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.977871+0000","last_change":"2024-04-25T11:39:22.977904+0000","last_active":"2024-04-25T11:39:22.977871+0000","last_peered":"2024-04-25T11:39:22.977871+0000","last_clean":"2024-04-25T11:39:22.977871+0000","last_became_active":"2024-04-25T11:38:25.666160+0000","last_became_peered":"2024-04-25T11:38:25.666160+0000","last_unstale":"2024-04-25T11:39:22.977871+0000","last_undegraded":"2024-04-25T11:39:22.977871+0000","last_fullsized":"2024-04-25T11:39:22.977871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.977782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.977782+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.977782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:57.881453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'7","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.030271+0000","last_change":"2024-04-25T11:39:21.030300+0000","last_active":"2024-04-25T11:39:21.030271+0000","last_peered":"2024-04-25T11:39:21.030271+0000","last_clean":"2024-04-25T11:39:21.030271+0000","last_became_active":"2024-04-25T11:38:24.666418+0000","last_became_peered":"2024-04-25T11:38:24.666418+0000","last_unstale":"2024-04-25T11:39:21.030271+0000","last_undegraded":"2024-04-25T11:39:21.030271+0000","last_fullsized":"2024-04-25T11:39:21.030271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T11:39:21.030186+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T11:39:21.030186+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.030186+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:21.131544+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.902500+0000","last_change":"2024-04-25T11:39:26.902526+0000","last_active":"2024-04-25T11:39:26.902500+0000","last_peered":"2024-04-25T11:39:26.902500+0000","last_clean":"2024-04-25T11:39:26.902500+0000","last_became_active":"2024-04-25T11:38:25.679725+0000","last_became_peered":"2024-04-25T11:38:25.679725+0000","last_unstale":"2024-04-25T11:39:26.902500+0000","last_undegraded":"2024-04-25T11:39:26.902500+0000","last_fullsized":"2024-04-25T11:39:26.902500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:26.902413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:26.902413+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.902413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:27.149460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'5","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.941088+0000","last_change":"2024-04-25T11:39:21.941124+0000","last_active":"2024-04-25T11:39:21.941088+0000","last_peered":"2024-04-25T11:39:21.941088+0000","last_clean":"2024-04-25T11:39:21.941088+0000","last_became_active":"2024-04-25T11:38:24.671247+0000","last_became_peered":"2024-04-25T11:38:24.671247+0000","last_unstale":"2024-04-25T11:39:21.941088+0000","last_undegraded":"2024-04-25T11:39:21.941088+0000","last_fullsized":"2024-04-25T11:39:21.941088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5","last_scrub_stamp":"2024-04-25T11:39:21.940974+0000","last_deep_scrub":"22'5","last_deep_scrub_stamp":"2024-04-25T11:39:21.940974+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.940974+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:34.470329+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.018506+0000","last_change":"2024-04-25T11:39:21.018545+0000","last_active":"2024-04-25T11:39:21.018506+0000","last_peered":"2024-04-25T11:39:21.018506+0000","last_clean":"2024-04-25T11:39:21.018506+0000","last_became_active":"2024-04-25T11:38:25.676495+0000","last_became_peered":"2024-04-25T11:38:25.676495+0000","last_unstale":"2024-04-25T11:39:21.018506+0000","last_undegraded":"2024-04-25T11:39:21.018506+0000","last_fullsized":"2024-04-25T11:39:21.018506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.018409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.018409+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.018409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:30.555823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'9","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.067256+0000","last_change":"2024-04-25T11:39:20.067289+0000","last_active":"2024-04-25T11:39:20.067256+0000","last_peered":"2024-04-25T11:39:20.067256+0000","last_clean":"2024-04-25T11:39:20.067256+0000","last_became_active":"2024-04-25T11:38:24.667003+0000","last_became_peered":"2024-04-25T11:38:24.667003+0000","last_unstale":"2024-04-25T11:39:20.067256+0000","last_undegraded":"2024-04-25T11:39:20.067256+0000","last_fullsized":"2024-04-25T11:39:20.067256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T11:39:20.067172+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T11:39:20.067172+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.067172+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:21.741738+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.985560+0000","last_change":"2024-04-25T11:39:20.985585+0000","last_active":"2024-04-25T11:39:20.985560+0000","last_peered":"2024-04-25T11:39:20.985560+0000","last_clean":"2024-04-25T11:39:20.985560+0000","last_became_active":"2024-04-25T11:38:25.680102+0000","last_became_peered":"2024-04-25T11:38:25.680102+0000","last_unstale":"2024-04-25T11:39:20.985560+0000","last_undegraded":"2024-04-25T11:39:20.985560+0000","last_fullsized":"2024-04-25T11:39:20.985560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:20.985469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:20.985469+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.985469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:27.741001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'17","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.080682+0000","last_change":"2024-04-25T11:39:20.080721+0000","last_active":"2024-04-25T11:39:20.080682+0000","last_peered":"2024-04-25T11:39:20.080682+0000","last_clean":"2024-04-25T11:39:20.080682+0000","last_became_active":"2024-04-25T11:38:24.662075+0000","last_became_peered":"2024-04-25T11:38:24.662075+0000","last_unstale":"2024-04-25T11:39:20.080682+0000","last_undegraded":"2024-04-25T11:39:20.080682+0000","last_fullsized":"2024-04-25T11:39:20.080682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'17","last_scrub_stamp":"2024-04-25T11:39:20.080598+0000","last_deep_scrub":"22'17","last_deep_scrub_stamp":"2024-04-25T11:39:20.080598+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.080598+0000","objects_scrubbed":6,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:26.225225+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3009,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:27.632332+0000","last_change":"2024-04-25T11:39:27.632375+0000","last_active":"2024-04-25T11:39:27.632332+0000","last_peered":"2024-04-25T11:39:27.632332+0000","last_clean":"2024-04-25T11:39:27.632332+0000","last_became_active":"2024-04-25T11:38:25.677096+0000","last_became_peered":"2024-04-25T11:38:25.677096+0000","last_unstale":"2024-04-25T11:39:27.632332+0000","last_undegraded":"2024-04-25T11:39:27.632332+0000","last_fullsized":"2024-04-25T11:39:27.632332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:27.632221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:27.632221+0000","last_clean_scrub_stamp":"2024-04-25T11:39:27.632221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:46.048609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'13","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:14.948141+0000","last_change":"2024-04-25T11:39:14.948195+0000","last_active":"2024-04-25T11:39:14.948141+0000","last_peered":"2024-04-25T11:39:14.948141+0000","last_clean":"2024-04-25T11:39:14.948141+0000","last_became_active":"2024-04-25T11:38:24.658673+0000","last_became_peered":"2024-04-25T11:38:24.658673+0000","last_unstale":"2024-04-25T11:39:14.948141+0000","last_undegraded":"2024-04-25T11:39:14.948141+0000","last_fullsized":"2024-04-25T11:39:14.948141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'13","last_scrub_stamp":"2024-04-25T11:39:14.948025+0000","last_deep_scrub":"22'13","last_deep_scrub_stamp":"2024-04-25T11:39:14.948025+0000","last_clean_scrub_stamp":"2024-04-25T11:39:14.948025+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:47.555902+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.952531+0000","last_change":"2024-04-25T11:39:23.952557+0000","last_active":"2024-04-25T11:39:23.952531+0000","last_peered":"2024-04-25T11:39:23.952531+0000","last_clean":"2024-04-25T11:39:23.952531+0000","last_became_active":"2024-04-25T11:38:25.681065+0000","last_became_peered":"2024-04-25T11:38:25.681065+0000","last_unstale":"2024-04-25T11:39:23.952531+0000","last_undegraded":"2024-04-25T11:39:23.952531+0000","last_fullsized":"2024-04-25T11:39:23.952531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.952446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.952446+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.952446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:16.212787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'7","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.907650+0000","last_change":"2024-04-25T11:39:18.907684+0000","last_active":"2024-04-25T11:39:18.907650+0000","last_peered":"2024-04-25T11:39:18.907650+0000","last_clean":"2024-04-25T11:39:18.907650+0000","last_became_active":"2024-04-25T11:38:24.666430+0000","last_became_peered":"2024-04-25T11:38:24.666430+0000","last_unstale":"2024-04-25T11:39:18.907650+0000","last_undegraded":"2024-04-25T11:39:18.907650+0000","last_fullsized":"2024-04-25T11:39:18.907650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T11:39:18.907564+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T11:39:18.907564+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.907564+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:43.024853+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:35.630143+0000","last_change":"2024-04-25T11:39:35.630167+0000","last_active":"2024-04-25T11:39:35.630143+0000","last_peered":"2024-04-25T11:39:35.630143+0000","last_clean":"2024-04-25T11:39:35.630143+0000","last_became_active":"2024-04-25T11:38:25.683781+0000","last_became_peered":"2024-04-25T11:38:25.683781+0000","last_unstale":"2024-04-25T11:39:35.630143+0000","last_undegraded":"2024-04-25T11:39:35.630143+0000","last_fullsized":"2024-04-25T11:39:35.630143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:35.630046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:35.630046+0000","last_clean_scrub_stamp":"2024-04-25T11:39:35.630046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:20.607917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'9","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.102937+0000","last_change":"2024-04-25T11:39:19.102998+0000","last_active":"2024-04-25T11:39:19.102937+0000","last_peered":"2024-04-25T11:39:19.102937+0000","last_clean":"2024-04-25T11:39:19.102937+0000","last_became_active":"2024-04-25T11:38:24.660265+0000","last_became_peered":"2024-04-25T11:38:24.660265+0000","last_unstale":"2024-04-25T11:39:19.102937+0000","last_undegraded":"2024-04-25T11:39:19.102937+0000","last_fullsized":"2024-04-25T11:39:19.102937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T11:39:19.102814+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T11:39:19.102814+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.102814+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:26.300866+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:29.114848+0000","last_change":"2024-04-25T11:39:29.114878+0000","last_active":"2024-04-25T11:39:29.114848+0000","last_peered":"2024-04-25T11:39:29.114848+0000","last_clean":"2024-04-25T11:39:29.114848+0000","last_became_active":"2024-04-25T11:38:25.679158+0000","last_became_peered":"2024-04-25T11:38:25.679158+0000","last_unstale":"2024-04-25T11:39:29.114848+0000","last_undegraded":"2024-04-25T11:39:29.114848+0000","last_fullsized":"2024-04-25T11:39:29.114848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:29.114745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:29.114745+0000","last_clean_scrub_stamp":"2024-04-25T11:39:29.114745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:22.370985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'25","reported_seq":52,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.804826+0000","last_change":"2024-04-25T11:39:18.804867+0000","last_active":"2024-04-25T11:39:18.804826+0000","last_peered":"2024-04-25T11:39:18.804826+0000","last_clean":"2024-04-25T11:39:18.804826+0000","last_became_active":"2024-04-25T11:38:24.664926+0000","last_became_peered":"2024-04-25T11:38:24.664926+0000","last_unstale":"2024-04-25T11:39:18.804826+0000","last_undegraded":"2024-04-25T11:39:18.804826+0000","last_fullsized":"2024-04-25T11:39:18.804826+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'25","last_scrub_stamp":"2024-04-25T11:39:18.804745+0000","last_deep_scrub":"22'25","last_deep_scrub_stamp":"2024-04-25T11:39:18.804745+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.804745+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:11.063457+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:33.649908+0000","last_change":"2024-04-25T11:39:33.649956+0000","last_active":"2024-04-25T11:39:33.649908+0000","last_peered":"2024-04-25T11:39:33.649908+0000","last_clean":"2024-04-25T11:39:33.649908+0000","last_became_active":"2024-04-25T11:38:25.681994+0000","last_became_peered":"2024-04-25T11:38:25.681994+0000","last_unstale":"2024-04-25T11:39:33.649908+0000","last_undegraded":"2024-04-25T11:39:33.649908+0000","last_fullsized":"2024-04-25T11:39:33.649908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:33.649794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:33.649794+0000","last_clean_scrub_stamp":"2024-04-25T11:39:33.649794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:43.704661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'3","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:16.975752+0000","last_change":"2024-04-25T11:39:16.975782+0000","last_active":"2024-04-25T11:39:16.975752+0000","last_peered":"2024-04-25T11:39:16.975752+0000","last_clean":"2024-04-25T11:39:16.975752+0000","last_became_active":"2024-04-25T11:38:24.658355+0000","last_became_peered":"2024-04-25T11:38:24.658355+0000","last_unstale":"2024-04-25T11:39:16.975752+0000","last_undegraded":"2024-04-25T11:39:16.975752+0000","last_fullsized":"2024-04-25T11:39:16.975752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2024-04-25T11:39:16.975657+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2024-04-25T11:39:16.975657+0000","last_clean_scrub_stamp":"2024-04-25T11:39:16.975657+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:17.312794+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:31.715555+0000","last_change":"2024-04-25T11:39:31.715582+0000","last_active":"2024-04-25T11:39:31.715555+0000","last_peered":"2024-04-25T11:39:31.715555+0000","last_clean":"2024-04-25T11:39:31.715555+0000","last_became_active":"2024-04-25T11:38:25.680493+0000","last_became_peered":"2024-04-25T11:38:25.680493+0000","last_unstale":"2024-04-25T11:39:31.715555+0000","last_undegraded":"2024-04-25T11:39:31.715555+0000","last_fullsized":"2024-04-25T11:39:31.715555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:31.715459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:31.715459+0000","last_clean_scrub_stamp":"2024-04-25T11:39:31.715459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:06.952171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'6","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.752493+0000","last_change":"2024-04-25T11:39:17.752537+0000","last_active":"2024-04-25T11:39:17.752493+0000","last_peered":"2024-04-25T11:39:17.752493+0000","last_clean":"2024-04-25T11:39:17.752493+0000","last_became_active":"2024-04-25T11:38:24.656479+0000","last_became_peered":"2024-04-25T11:38:24.656479+0000","last_unstale":"2024-04-25T11:39:17.752493+0000","last_undegraded":"2024-04-25T11:39:17.752493+0000","last_fullsized":"2024-04-25T11:39:17.752493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6","last_scrub_stamp":"2024-04-25T11:39:17.752385+0000","last_deep_scrub":"22'6","last_deep_scrub_stamp":"2024-04-25T11:39:17.752385+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.752385+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:10.789648+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:32.689067+0000","last_change":"2024-04-25T11:39:32.689093+0000","last_active":"2024-04-25T11:39:32.689067+0000","last_peered":"2024-04-25T11:39:32.689067+0000","last_clean":"2024-04-25T11:39:32.689067+0000","last_became_active":"2024-04-25T11:38:25.680940+0000","last_became_peered":"2024-04-25T11:38:25.680940+0000","last_unstale":"2024-04-25T11:39:32.689067+0000","last_undegraded":"2024-04-25T11:39:32.689067+0000","last_fullsized":"2024-04-25T11:39:32.689067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:32.688973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:32.688973+0000","last_clean_scrub_stamp":"2024-04-25T11:39:32.688973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:01.616854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'4","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.033205+0000","last_change":"2024-04-25T11:39:18.033230+0000","last_active":"2024-04-25T11:39:18.033205+0000","last_peered":"2024-04-25T11:39:18.033205+0000","last_clean":"2024-04-25T11:39:18.033205+0000","last_became_active":"2024-04-25T11:38:24.662947+0000","last_became_peered":"2024-04-25T11:38:24.662947+0000","last_unstale":"2024-04-25T11:39:18.033205+0000","last_undegraded":"2024-04-25T11:39:18.033205+0000","last_fullsized":"2024-04-25T11:39:18.033205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T11:39:18.033145+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T11:39:18.033145+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.033145+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:15.822794+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.893787+0000","last_change":"2024-04-25T11:39:23.893829+0000","last_active":"2024-04-25T11:39:23.893787+0000","last_peered":"2024-04-25T11:39:23.893787+0000","last_clean":"2024-04-25T11:39:23.893787+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:39:23.893787+0000","last_undegraded":"2024-04-25T11:39:23.893787+0000","last_fullsized":"2024-04-25T11:39:23.893787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:23.893704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:23.893704+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.893704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:10.187188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'9","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.758095+0000","last_change":"2024-04-25T11:39:19.758170+0000","last_active":"2024-04-25T11:39:19.758095+0000","last_peered":"2024-04-25T11:39:19.758095+0000","last_clean":"2024-04-25T11:39:19.758095+0000","last_became_active":"2024-04-25T11:38:24.666516+0000","last_became_peered":"2024-04-25T11:38:24.666516+0000","last_unstale":"2024-04-25T11:39:19.758095+0000","last_undegraded":"2024-04-25T11:39:19.758095+0000","last_fullsized":"2024-04-25T11:39:19.758095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'9","last_scrub_stamp":"2024-04-25T11:39:19.757978+0000","last_deep_scrub":"22'9","last_deep_scrub_stamp":"2024-04-25T11:39:19.757978+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.757978+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:52.365855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.619262+0000","last_change":"2024-04-25T11:39:18.619288+0000","last_active":"2024-04-25T11:39:18.619262+0000","last_peered":"2024-04-25T11:39:18.619262+0000","last_clean":"2024-04-25T11:39:18.619262+0000","last_became_active":"2024-04-25T11:38:25.677414+0000","last_became_peered":"2024-04-25T11:38:25.677414+0000","last_unstale":"2024-04-25T11:39:18.619262+0000","last_undegraded":"2024-04-25T11:39:18.619262+0000","last_fullsized":"2024-04-25T11:39:18.619262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:18.619164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:18.619164+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.619164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:02.362883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'119","reported_seq":184,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.605564+0000","last_change":"2024-04-25T11:39:20.605604+0000","last_active":"2024-04-25T11:39:20.605564+0000","last_peered":"2024-04-25T11:39:20.605564+0000","last_clean":"2024-04-25T11:39:20.605564+0000","last_became_active":"2024-04-25T11:38:20.864860+0000","last_became_peered":"2024-04-25T11:38:20.864860+0000","last_unstale":"2024-04-25T11:39:20.605564+0000","last_undegraded":"2024-04-25T11:39:20.605564+0000","last_fullsized":"2024-04-25T11:39:20.605564+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'119","last_scrub_stamp":"2024-04-25T11:39:20.605477+0000","last_deep_scrub":"22'119","last_deep_scrub_stamp":"2024-04-25T11:39:20.605477+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.605477+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:24.126153+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'10","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:20.909392+0000","last_change":"2024-04-25T11:39:20.909430+0000","last_active":"2024-04-25T11:39:20.909392+0000","last_peered":"2024-04-25T11:39:20.909392+0000","last_clean":"2024-04-25T11:39:20.909392+0000","last_became_active":"2024-04-25T11:38:24.670356+0000","last_became_peered":"2024-04-25T11:38:24.670356+0000","last_unstale":"2024-04-25T11:39:20.909392+0000","last_undegraded":"2024-04-25T11:39:20.909392+0000","last_fullsized":"2024-04-25T11:39:20.909392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'10","last_scrub_stamp":"2024-04-25T11:39:20.909304+0000","last_deep_scrub":"22'10","last_deep_scrub_stamp":"2024-04-25T11:39:20.909304+0000","last_clean_scrub_stamp":"2024-04-25T11:39:20.909304+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:07.398793+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.947123+0000","last_change":"2024-04-25T11:39:24.947155+0000","last_active":"2024-04-25T11:39:24.947123+0000","last_peered":"2024-04-25T11:39:24.947123+0000","last_clean":"2024-04-25T11:39:24.947123+0000","last_became_active":"2024-04-25T11:38:25.674698+0000","last_became_peered":"2024-04-25T11:38:25.674698+0000","last_unstale":"2024-04-25T11:39:24.947123+0000","last_undegraded":"2024-04-25T11:39:24.947123+0000","last_fullsized":"2024-04-25T11:39:24.947123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.947040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.947040+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.947040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:04.554313+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'5","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.606611+0000","last_change":"2024-04-25T11:39:24.606646+0000","last_active":"2024-04-25T11:39:24.606611+0000","last_peered":"2024-04-25T11:39:24.606611+0000","last_clean":"2024-04-25T11:39:24.606611+0000","last_became_active":"2024-04-25T11:38:24.670819+0000","last_became_peered":"2024-04-25T11:38:24.670819+0000","last_unstale":"2024-04-25T11:39:24.606611+0000","last_undegraded":"2024-04-25T11:39:24.606611+0000","last_fullsized":"2024-04-25T11:39:24.606611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5","last_scrub_stamp":"2024-04-25T11:39:24.606523+0000","last_deep_scrub":"22'5","last_deep_scrub_stamp":"2024-04-25T11:39:24.606523+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.606523+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:27.509086+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:16.965777+0000","last_change":"2024-04-25T11:39:16.965816+0000","last_active":"2024-04-25T11:39:16.965777+0000","last_peered":"2024-04-25T11:39:16.965777+0000","last_clean":"2024-04-25T11:39:16.965777+0000","last_became_active":"2024-04-25T11:38:25.673056+0000","last_became_peered":"2024-04-25T11:38:25.673056+0000","last_unstale":"2024-04-25T11:39:16.965777+0000","last_undegraded":"2024-04-25T11:39:16.965777+0000","last_fullsized":"2024-04-25T11:39:16.965777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:16.965687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:16.965687+0000","last_clean_scrub_stamp":"2024-04-25T11:39:16.965687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:35.135027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'7","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:16.733625+0000","last_change":"2024-04-25T11:39:16.733652+0000","last_active":"2024-04-25T11:39:16.733625+0000","last_peered":"2024-04-25T11:39:16.733625+0000","last_clean":"2024-04-25T11:39:16.733625+0000","last_became_active":"2024-04-25T11:38:24.650266+0000","last_became_peered":"2024-04-25T11:38:24.650266+0000","last_unstale":"2024-04-25T11:39:16.733625+0000","last_undegraded":"2024-04-25T11:39:16.733625+0000","last_fullsized":"2024-04-25T11:39:16.733625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T11:39:16.733554+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T11:39:16.733554+0000","last_clean_scrub_stamp":"2024-04-25T11:39:16.733554+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:17.729453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.017550+0000","last_change":"2024-04-25T11:39:24.017572+0000","last_active":"2024-04-25T11:39:24.017550+0000","last_peered":"2024-04-25T11:39:24.017550+0000","last_clean":"2024-04-25T11:39:24.017550+0000","last_became_active":"2024-04-25T11:38:25.668624+0000","last_became_peered":"2024-04-25T11:38:25.668624+0000","last_unstale":"2024-04-25T11:39:24.017550+0000","last_undegraded":"2024-04-25T11:39:24.017550+0000","last_fullsized":"2024-04-25T11:39:24.017550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.017466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.017466+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.017466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:48.134755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.918845+0000","last_change":"2024-04-25T11:39:17.918908+0000","last_active":"2024-04-25T11:39:17.918845+0000","last_peered":"2024-04-25T11:39:17.918845+0000","last_clean":"2024-04-25T11:39:17.918845+0000","last_became_active":"2024-04-25T11:38:24.664745+0000","last_became_peered":"2024-04-25T11:38:24.664745+0000","last_unstale":"2024-04-25T11:39:17.918845+0000","last_undegraded":"2024-04-25T11:39:17.918845+0000","last_fullsized":"2024-04-25T11:39:17.918845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'5","last_scrub_stamp":"2024-04-25T11:39:17.918726+0000","last_deep_scrub":"20'5","last_deep_scrub_stamp":"2024-04-25T11:39:17.918726+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.918726+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:52.822397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.940077+0000","last_change":"2024-04-25T11:39:21.940117+0000","last_active":"2024-04-25T11:39:21.940077+0000","last_peered":"2024-04-25T11:39:21.940077+0000","last_clean":"2024-04-25T11:39:21.940077+0000","last_became_active":"2024-04-25T11:38:25.681498+0000","last_became_peered":"2024-04-25T11:38:25.681498+0000","last_unstale":"2024-04-25T11:39:21.940077+0000","last_undegraded":"2024-04-25T11:39:21.940077+0000","last_fullsized":"2024-04-25T11:39:21.940077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.939941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.939941+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.939941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.137993+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":28,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:15.967043+0000","last_change":"2024-04-25T11:39:15.967092+0000","last_active":"2024-04-25T11:39:15.967043+0000","last_peered":"2024-04-25T11:39:15.967043+0000","last_clean":"2024-04-25T11:39:15.967043+0000","last_became_active":"2024-04-25T11:38:24.659450+0000","last_became_peered":"2024-04-25T11:38:24.659450+0000","last_unstale":"2024-04-25T11:39:15.967043+0000","last_undegraded":"2024-04-25T11:39:15.967043+0000","last_fullsized":"2024-04-25T11:39:15.967043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-04-25T11:39:15.966956+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-04-25T11:39:15.966956+0000","last_clean_scrub_stamp":"2024-04-25T11:39:15.966956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:08.225668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.725955+0000","last_change":"2024-04-25T11:39:22.725981+0000","last_active":"2024-04-25T11:39:22.725955+0000","last_peered":"2024-04-25T11:39:22.725955+0000","last_clean":"2024-04-25T11:39:22.725955+0000","last_became_active":"2024-04-25T11:39:14.114913+0000","last_became_peered":"2024-04-25T11:39:14.114913+0000","last_unstale":"2024-04-25T11:39:22.725955+0000","last_undegraded":"2024-04-25T11:39:22.725955+0000","last_fullsized":"2024-04-25T11:39:22.725955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.725863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.725863+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.725863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:22.827221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"22'7","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.609388+0000","last_change":"2024-04-25T11:39:25.609424+0000","last_active":"2024-04-25T11:39:25.609388+0000","last_peered":"2024-04-25T11:39:25.609388+0000","last_clean":"2024-04-25T11:39:25.609388+0000","last_became_active":"2024-04-25T11:38:24.671120+0000","last_became_peered":"2024-04-25T11:38:24.671120+0000","last_unstale":"2024-04-25T11:39:25.609388+0000","last_undegraded":"2024-04-25T11:39:25.609388+0000","last_fullsized":"2024-04-25T11:39:25.609388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7","last_scrub_stamp":"2024-04-25T11:39:25.609295+0000","last_deep_scrub":"22'7","last_deep_scrub_stamp":"2024-04-25T11:39:25.609295+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.609295+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:40.145347+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.999448+0000","last_change":"2024-04-25T11:39:24.999473+0000","last_active":"2024-04-25T11:39:24.999448+0000","last_peered":"2024-04-25T11:39:24.999448+0000","last_clean":"2024-04-25T11:39:24.999448+0000","last_became_active":"2024-04-25T11:38:25.668662+0000","last_became_peered":"2024-04-25T11:38:25.668662+0000","last_unstale":"2024-04-25T11:39:24.999448+0000","last_undegraded":"2024-04-25T11:39:24.999448+0000","last_fullsized":"2024-04-25T11:39:24.999448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.999360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.999360+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.999360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:28.520036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'11","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:19.068245+0000","last_change":"2024-04-25T11:39:19.068279+0000","last_active":"2024-04-25T11:39:19.068245+0000","last_peered":"2024-04-25T11:39:19.068245+0000","last_clean":"2024-04-25T11:39:19.068245+0000","last_became_active":"2024-04-25T11:38:24.663863+0000","last_became_peered":"2024-04-25T11:38:24.663863+0000","last_unstale":"2024-04-25T11:39:19.068245+0000","last_undegraded":"2024-04-25T11:39:19.068245+0000","last_fullsized":"2024-04-25T11:39:19.068245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'11","last_scrub_stamp":"2024-04-25T11:39:19.068161+0000","last_deep_scrub":"22'11","last_deep_scrub_stamp":"2024-04-25T11:39:19.068161+0000","last_clean_scrub_stamp":"2024-04-25T11:39:19.068161+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:19.169520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.984280+0000","last_change":"2024-04-25T11:39:25.984305+0000","last_active":"2024-04-25T11:39:25.984280+0000","last_peered":"2024-04-25T11:39:25.984280+0000","last_clean":"2024-04-25T11:39:25.984280+0000","last_became_active":"2024-04-25T11:38:25.679774+0000","last_became_peered":"2024-04-25T11:38:25.679774+0000","last_unstale":"2024-04-25T11:39:25.984280+0000","last_undegraded":"2024-04-25T11:39:25.984280+0000","last_fullsized":"2024-04-25T11:39:25.984280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:25.984184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:25.984184+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.984184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:41.271682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'10","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.057331+0000","last_change":"2024-04-25T11:39:22.057365+0000","last_active":"2024-04-25T11:39:22.057331+0000","last_peered":"2024-04-25T11:39:22.057331+0000","last_clean":"2024-04-25T11:39:22.057331+0000","last_became_active":"2024-04-25T11:38:24.668054+0000","last_became_peered":"2024-04-25T11:38:24.668054+0000","last_unstale":"2024-04-25T11:39:22.057331+0000","last_undegraded":"2024-04-25T11:39:22.057331+0000","last_fullsized":"2024-04-25T11:39:22.057331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'10","last_scrub_stamp":"2024-04-25T11:39:22.057233+0000","last_deep_scrub":"22'10","last_deep_scrub_stamp":"2024-04-25T11:39:22.057233+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.057233+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:23.731799+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.931002+0000","last_change":"2024-04-25T11:39:17.931022+0000","last_active":"2024-04-25T11:39:17.931002+0000","last_peered":"2024-04-25T11:39:17.931002+0000","last_clean":"2024-04-25T11:39:17.931002+0000","last_became_active":"2024-04-25T11:38:25.676134+0000","last_became_peered":"2024-04-25T11:38:25.676134+0000","last_unstale":"2024-04-25T11:39:17.931002+0000","last_undegraded":"2024-04-25T11:39:17.931002+0000","last_fullsized":"2024-04-25T11:39:17.931002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:17.930937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:17.930937+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.930937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:47.988266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'3","reported_seq":30,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.622937+0000","last_change":"2024-04-25T11:39:26.622972+0000","last_active":"2024-04-25T11:39:26.622937+0000","last_peered":"2024-04-25T11:39:26.622937+0000","last_clean":"2024-04-25T11:39:26.622937+0000","last_became_active":"2024-04-25T11:38:24.671657+0000","last_became_peered":"2024-04-25T11:38:24.671657+0000","last_unstale":"2024-04-25T11:39:26.622937+0000","last_undegraded":"2024-04-25T11:39:26.622937+0000","last_fullsized":"2024-04-25T11:39:26.622937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2024-04-25T11:39:26.622844+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2024-04-25T11:39:26.622844+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.622844+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:59.922932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.954801+0000","last_change":"2024-04-25T11:39:18.954835+0000","last_active":"2024-04-25T11:39:18.954801+0000","last_peered":"2024-04-25T11:39:18.954801+0000","last_clean":"2024-04-25T11:39:18.954801+0000","last_became_active":"2024-04-25T11:38:25.669178+0000","last_became_peered":"2024-04-25T11:38:25.669178+0000","last_unstale":"2024-04-25T11:39:18.954801+0000","last_undegraded":"2024-04-25T11:39:18.954801+0000","last_fullsized":"2024-04-25T11:39:18.954801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:18.954717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:18.954717+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.954717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:04.008478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'10","reported_seq":37,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.027729+0000","last_change":"2024-04-25T11:39:21.027774+0000","last_active":"2024-04-25T11:39:21.027729+0000","last_peered":"2024-04-25T11:39:21.027729+0000","last_clean":"2024-04-25T11:39:21.027729+0000","last_became_active":"2024-04-25T11:38:24.668420+0000","last_became_peered":"2024-04-25T11:38:24.668420+0000","last_unstale":"2024-04-25T11:39:21.027729+0000","last_undegraded":"2024-04-25T11:39:21.027729+0000","last_fullsized":"2024-04-25T11:39:21.027729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'10","last_scrub_stamp":"2024-04-25T11:39:21.027636+0000","last_deep_scrub":"22'10","last_deep_scrub_stamp":"2024-04-25T11:39:21.027636+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.027636+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:25.562826+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.013764+0000","last_change":"2024-04-25T11:39:22.013788+0000","last_active":"2024-04-25T11:39:22.013764+0000","last_peered":"2024-04-25T11:39:22.013764+0000","last_clean":"2024-04-25T11:39:22.013764+0000","last_became_active":"2024-04-25T11:38:25.667778+0000","last_became_peered":"2024-04-25T11:38:25.667778+0000","last_unstale":"2024-04-25T11:39:22.013764+0000","last_undegraded":"2024-04-25T11:39:22.013764+0000","last_fullsized":"2024-04-25T11:39:22.013764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:22.013664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:22.013664+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.013664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:46.130953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'4","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:17.012071+0000","last_change":"2024-04-25T11:39:17.012096+0000","last_active":"2024-04-25T11:39:17.012071+0000","last_peered":"2024-04-25T11:39:17.012071+0000","last_clean":"2024-04-25T11:39:17.012071+0000","last_became_active":"2024-04-25T11:38:24.662539+0000","last_became_peered":"2024-04-25T11:38:24.662539+0000","last_unstale":"2024-04-25T11:39:17.012071+0000","last_undegraded":"2024-04-25T11:39:17.012071+0000","last_fullsized":"2024-04-25T11:39:17.012071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2024-04-25T11:39:17.012002+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2024-04-25T11:39:17.012002+0000","last_clean_scrub_stamp":"2024-04-25T11:39:17.012002+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:49.619879+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:26.967162+0000","last_change":"2024-04-25T11:39:26.967188+0000","last_active":"2024-04-25T11:39:26.967162+0000","last_peered":"2024-04-25T11:39:26.967162+0000","last_clean":"2024-04-25T11:39:26.967162+0000","last_became_active":"2024-04-25T11:38:25.679838+0000","last_became_peered":"2024-04-25T11:38:25.679838+0000","last_unstale":"2024-04-25T11:39:26.967162+0000","last_undegraded":"2024-04-25T11:39:26.967162+0000","last_fullsized":"2024-04-25T11:39:26.967162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:26.967076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:26.967076+0000","last_clean_scrub_stamp":"2024-04-25T11:39:26.967076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.513286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'6","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.584634+0000","last_change":"2024-04-25T11:39:21.584667+0000","last_active":"2024-04-25T11:39:21.584634+0000","last_peered":"2024-04-25T11:39:21.584634+0000","last_clean":"2024-04-25T11:39:21.584634+0000","last_became_active":"2024-04-25T11:38:24.668429+0000","last_became_peered":"2024-04-25T11:38:24.668429+0000","last_unstale":"2024-04-25T11:39:21.584634+0000","last_undegraded":"2024-04-25T11:39:21.584634+0000","last_fullsized":"2024-04-25T11:39:21.584634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6","last_scrub_stamp":"2024-04-25T11:39:21.584550+0000","last_deep_scrub":"22'6","last_deep_scrub_stamp":"2024-04-25T11:39:21.584550+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.584550+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:34.113905+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:24.078553+0000","last_change":"2024-04-25T11:39:24.078579+0000","last_active":"2024-04-25T11:39:24.078553+0000","last_peered":"2024-04-25T11:39:24.078553+0000","last_clean":"2024-04-25T11:39:24.078553+0000","last_became_active":"2024-04-25T11:38:25.668099+0000","last_became_peered":"2024-04-25T11:38:25.668099+0000","last_unstale":"2024-04-25T11:39:24.078553+0000","last_undegraded":"2024-04-25T11:39:24.078553+0000","last_fullsized":"2024-04-25T11:39:24.078553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:24.078465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:24.078465+0000","last_clean_scrub_stamp":"2024-04-25T11:39:24.078465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.615880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:30.755577+0000","last_change":"2024-04-25T11:39:30.755602+0000","last_active":"2024-04-25T11:39:30.755577+0000","last_peered":"2024-04-25T11:39:30.755577+0000","last_clean":"2024-04-25T11:39:30.755577+0000","last_became_active":"2024-04-25T11:38:25.683362+0000","last_became_peered":"2024-04-25T11:38:25.683362+0000","last_unstale":"2024-04-25T11:39:30.755577+0000","last_undegraded":"2024-04-25T11:39:30.755577+0000","last_fullsized":"2024-04-25T11:39:30.755577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:30.755487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:30.755487+0000","last_clean_scrub_stamp":"2024-04-25T11:39:30.755487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:05.143451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'8","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:23.576560+0000","last_change":"2024-04-25T11:39:23.576596+0000","last_active":"2024-04-25T11:39:23.576560+0000","last_peered":"2024-04-25T11:39:23.576560+0000","last_clean":"2024-04-25T11:39:23.576560+0000","last_became_active":"2024-04-25T11:38:24.670038+0000","last_became_peered":"2024-04-25T11:38:24.670038+0000","last_unstale":"2024-04-25T11:39:23.576560+0000","last_undegraded":"2024-04-25T11:39:23.576560+0000","last_fullsized":"2024-04-25T11:39:23.576560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'8","last_scrub_stamp":"2024-04-25T11:39:23.576477+0000","last_deep_scrub":"22'8","last_deep_scrub_stamp":"2024-04-25T11:39:23.576477+0000","last_clean_scrub_stamp":"2024-04-25T11:39:23.576477+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:38.863975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:28.833323+0000","last_change":"2024-04-25T11:39:28.833348+0000","last_active":"2024-04-25T11:39:28.833323+0000","last_peered":"2024-04-25T11:39:28.833323+0000","last_clean":"2024-04-25T11:39:28.833323+0000","last_became_active":"2024-04-25T11:38:25.678570+0000","last_became_peered":"2024-04-25T11:38:25.678570+0000","last_unstale":"2024-04-25T11:39:28.833323+0000","last_undegraded":"2024-04-25T11:39:28.833323+0000","last_fullsized":"2024-04-25T11:39:28.833323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:28.833225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:28.833225+0000","last_clean_scrub_stamp":"2024-04-25T11:39:28.833225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:19.571213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'16","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:22.610400+0000","last_change":"2024-04-25T11:39:22.610438+0000","last_active":"2024-04-25T11:39:22.610400+0000","last_peered":"2024-04-25T11:39:22.610400+0000","last_clean":"2024-04-25T11:39:22.610400+0000","last_became_active":"2024-04-25T11:38:24.669175+0000","last_became_peered":"2024-04-25T11:38:24.669175+0000","last_unstale":"2024-04-25T11:39:22.610400+0000","last_undegraded":"2024-04-25T11:39:22.610400+0000","last_fullsized":"2024-04-25T11:39:22.610400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'16","last_scrub_stamp":"2024-04-25T11:39:22.610318+0000","last_deep_scrub":"22'16","last_deep_scrub_stamp":"2024-04-25T11:39:22.610318+0000","last_clean_scrub_stamp":"2024-04-25T11:39:22.610318+0000","objects_scrubbed":7,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:15.866558+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:25.974992+0000","last_change":"2024-04-25T11:39:25.975058+0000","last_active":"2024-04-25T11:39:25.974992+0000","last_peered":"2024-04-25T11:39:25.974992+0000","last_clean":"2024-04-25T11:39:25.974992+0000","last_became_active":"2024-04-25T11:38:25.684188+0000","last_became_peered":"2024-04-25T11:38:25.684188+0000","last_unstale":"2024-04-25T11:39:25.974992+0000","last_undegraded":"2024-04-25T11:39:25.974992+0000","last_fullsized":"2024-04-25T11:39:25.974992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:25.974874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:25.974874+0000","last_clean_scrub_stamp":"2024-04-25T11:39:25.974874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:23.557971+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:18.014789+0000","last_change":"2024-04-25T11:39:18.014815+0000","last_active":"2024-04-25T11:39:18.014789+0000","last_peered":"2024-04-25T11:39:18.014789+0000","last_clean":"2024-04-25T11:39:18.014789+0000","last_became_active":"2024-04-25T11:38:24.658423+0000","last_became_peered":"2024-04-25T11:38:24.658423+0000","last_unstale":"2024-04-25T11:39:18.014789+0000","last_undegraded":"2024-04-25T11:39:18.014789+0000","last_fullsized":"2024-04-25T11:39:18.014789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'2","last_scrub_stamp":"2024-04-25T11:39:18.014675+0000","last_deep_scrub":"20'2","last_deep_scrub_stamp":"2024-04-25T11:39:18.014675+0000","last_clean_scrub_stamp":"2024-04-25T11:39:18.014675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:19.689241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:21.024589+0000","last_change":"2024-04-25T11:39:21.024616+0000","last_active":"2024-04-25T11:39:21.024589+0000","last_peered":"2024-04-25T11:39:21.024589+0000","last_clean":"2024-04-25T11:39:21.024589+0000","last_became_active":"2024-04-25T11:38:25.667195+0000","last_became_peered":"2024-04-25T11:38:25.667195+0000","last_unstale":"2024-04-25T11:39:21.024589+0000","last_undegraded":"2024-04-25T11:39:21.024589+0000","last_fullsized":"2024-04-25T11:39:21.024589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:39:21.024497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:39:21.024497+0000","last_clean_scrub_stamp":"2024-04-25T11:39:21.024497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:55.928168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'17","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:39:15.948303+0000","last_change":"2024-04-25T11:39:15.948334+0000","last_active":"2024-04-25T11:39:15.948303+0000","last_peered":"2024-04-25T11:39:15.948303+0000","last_clean":"2024-04-25T11:39:15.948303+0000","last_became_active":"2024-04-25T11:38:24.668999+0000","last_became_peered":"2024-04-25T11:38:24.668999+0000","last_unstale":"2024-04-25T11:39:15.948303+0000","last_undegraded":"2024-04-25T11:39:15.948303+0000","last_fullsized":"2024-04-25T11:39:15.948303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'17","last_scrub_stamp":"2024-04-25T11:39:15.948234+0000","last_deep_scrub":"22'17","last_deep_scrub_stamp":"2024-04-25T11:39:15.948234+0000","last_clean_scrub_stamp":"2024-04-25T11:39:15.948234+0000","objects_scrubbed":6,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:23.146286+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11177,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":218,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":22354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64736,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":331776,"data_stored":676166,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":87497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":87630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":85028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":87531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":85160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":258048,"data_stored":84648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":323584,"data_stored":674358,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32368,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:39:59.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T11:39:59.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.164+0000 7f7ecdc59640 1 Processor -- start 2024-04-25T11:39:59.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.164+0000 7f7ecdc59640 1 -- start start 2024-04-25T11:39:59.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.165+0000 7f7ecdc59640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 0x7f7ec80c5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.165+0000 7f7ecdc59640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ec815e3a0 0x7f7ec815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.165+0000 7f7ecdc59640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec806d7b0 con 0x7f7ec80cc700 2024-04-25T11:39:59.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.165+0000 7f7ecdc59640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7ec8155400 con 0x7f7ec815e3a0 2024-04-25T11:39:59.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.165+0000 7f7ecdc59640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec80c5e40 con 0x7f7ec8155020 2024-04-25T11:39:59.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.165+0000 7f7ec7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ec815e3a0 0x7f7ec815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.165+0000 7f7ec6ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7ec80cc700 0x7f7ec80cb450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57172/0 (socket says 172.21.15.40:57172) 2024-04-25T11:39:59.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.165+0000 7f7ec6ffd640 1 -- 172.21.15.40:0/2633070694 learned_addr learned my addr 172.21.15.40:0/2633070694 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:59.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.165+0000 7f7ec77fe640 1 --2- 172.21.15.40:0/2633070694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 0x7f7ec80c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.166+0000 7f7ec7fff640 1 -- 172.21.15.40:0/2633070694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 msgr2=0x7f7ec80c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:59.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.166+0000 7f7ec7fff640 1 --2- 172.21.15.40:0/2633070694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 0x7f7ec80c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.166+0000 7f7ec7fff640 1 -- 172.21.15.40:0/2633070694 >> v1:172.21.15.40:6789/0 conn(0x7f7ec80cc700 legacy=0x7f7ec80cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.166+0000 7f7ec7fff640 1 -- 172.21.15.40:0/2633070694 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec815ae70 con 0x7f7ec815e3a0 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.166+0000 7f7ec7fff640 1 --2- 172.21.15.40:0/2633070694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec815e3a0 0x7f7ec815e7b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009d90 tx=0x7f7ec002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a587d79e5cc1d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.166+0000 7f7ec67fc640 1 -- 172.21.15.40:0/2633070694 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0007740 con 0x7f7ec815e3a0 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ec67fc640 1 -- 172.21.15.40:0/2633070694 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec00078e0 con 0x7f7ec815e3a0 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ec67fc640 1 -- 172.21.15.40:0/2633070694 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0007cb0 con 0x7f7ec815e3a0 2024-04-25T11:39:59.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ecdc59640 1 -- 172.21.15.40:0/2633070694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec815e3a0 msgr2=0x7f7ec815e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ecdc59640 1 --2- 172.21.15.40:0/2633070694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec815e3a0 0x7f7ec815e7b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009d90 tx=0x7f7ec002a660 comp rx=0 tx=0).stop 2024-04-25T11:39:59.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ecdc59640 1 -- 172.21.15.40:0/2633070694 shutdown_connections 2024-04-25T11:39:59.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ecdc59640 1 --2- 172.21.15.40:0/2633070694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec815e3a0 0x7f7ec815e7b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ecdc59640 1 --2- 172.21.15.40:0/2633070694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 0x7f7ec80c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ecdc59640 1 -- 172.21.15.40:0/2633070694 >> 172.21.15.40:0/2633070694 conn(0x7f7ec8067890 msgr2=0x7f7ec8067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ecdc59640 1 -- 172.21.15.40:0/2633070694 shutdown_connections 2024-04-25T11:39:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.167+0000 7f7ecdc59640 1 -- 172.21.15.40:0/2633070694 wait complete. 2024-04-25T11:39:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.168+0000 7f7ecdc59640 1 Processor -- start 2024-04-25T11:39:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ecdc59640 1 -- start start 2024-04-25T11:39:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ecdc59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec80cc700 0x7f7ec815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ecdc59640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 0x7f7ec814a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ecdc59640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ec815e3a0 0x7f7ec814a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ecdc59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec815b710 con 0x7f7ec80cc700 2024-04-25T11:39:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ecdc59640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec815b430 con 0x7f7ec815e3a0 2024-04-25T11:39:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ecdc59640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec815ecf0 con 0x7f7ec8155020 2024-04-25T11:39:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ec77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec80cc700 0x7f7ec815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ec77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec80cc700 0x7f7ec815c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46784/0 (socket says 172.21.15.40:46784) 2024-04-25T11:39:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.169+0000 7f7ec77fe640 1 -- 172.21.15.40:0/766049830 learned_addr learned my addr 172.21.15.40:0/766049830 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ec6ffd640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 0x7f7ec814a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ec7fff640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ec815e3a0 0x7f7ec814a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ec77fe640 1 -- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 msgr2=0x7f7ec814a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ec77fe640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 0x7f7ec814a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ec77fe640 1 -- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ec815e3a0 msgr2=0x7f7ec814a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ec77fe640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ec815e3a0 0x7f7ec814a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ec77fe640 1 -- 172.21.15.40:0/766049830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec815dc70 con 0x7f7ec80cc700 2024-04-25T11:39:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ec77fe640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec80cc700 0x7f7ec815c980 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800e9b0 tx=0x7f7eb800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7eaffff640 1 -- 172.21.15.40:0/766049830 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8017070 con 0x7f7ec80cc700 2024-04-25T11:39:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7eaffff640 1 -- 172.21.15.40:0/766049830 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb800c9a0 con 0x7f7ec80cc700 2024-04-25T11:39:59.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec815df50 con 0x7f7ec80cc700 2024-04-25T11:39:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7eaffff640 1 -- 172.21.15.40:0/766049830 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb800cd90 con 0x7f7ec80cc700 2024-04-25T11:39:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.170+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec8149900 con 0x7f7ec80cc700 2024-04-25T11:39:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.171+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec8069200 con 0x7f7ec80cc700 2024-04-25T11:39:59.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.172+0000 7f7eaffff640 1 -- 172.21.15.40:0/766049830 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7eb801a070 con 0x7f7ec80cc700 2024-04-25T11:39:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.172+0000 7f7eaffff640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7eb406c8f0 0x7f7eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.172+0000 7f7eaffff640 1 -- 172.21.15.40:0/766049830 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7eb808bed0 con 0x7f7ec80cc700 2024-04-25T11:39:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.172+0000 7f7ec6ffd640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7eb406c8f0 0x7f7eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.173+0000 7f7ec6ffd640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7eb406c8f0 0x7f7eb406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80cbd70 tx=0x7f7ebc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.176+0000 7f7eaffff640 1 -- 172.21.15.40:0/766049830 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7eb805a310 con 0x7f7ec80cc700 2024-04-25T11:39:59.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.361+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f7ec80cbf90 con 0x7f7ec80cc700 2024-04-25T11:39:59.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.364+0000 7f7eaffff640 1 -- 172.21.15.40:0/766049830 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7f7eb8059c10 con 0x7f7ec80cc700 2024-04-25T11:39:59.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.364+0000 7f7eaffff640 1 -- 172.21.15.40:0/766049830 <== mon.0 v2:172.21.15.40:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7eb805dfc0 con 0x7f7ec80cc700 2024-04-25T11:39:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7eb406c8f0 msgr2=0x7f7eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7eb406c8f0 0x7f7eb406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80cbd70 tx=0x7f7ebc009040 comp rx=0 tx=0).stop 2024-04-25T11:39:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec80cc700 msgr2=0x7f7ec815c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec80cc700 0x7f7ec815c980 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800e9b0 tx=0x7f7eb800ee70 comp rx=0 tx=0).stop 2024-04-25T11:39:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 shutdown_connections 2024-04-25T11:39:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f7eb406c8f0 0x7f7eb406edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ec815e3a0 0x7f7ec814a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ec8155020 0x7f7ec814a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 --2- 172.21.15.40:0/766049830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec80cc700 0x7f7ec815c980 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 >> 172.21.15.40:0/766049830 conn(0x7f7ec8067890 msgr2=0x7f7ec8067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 shutdown_connections 2024-04-25T11:39:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.366+0000 7f7ecdc59640 1 -- 172.21.15.40:0/766049830 wait complete. 2024-04-25T11:39:59.364 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:39:59.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:59.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.478+0000 7f5b65f87640 1 Processor -- start 2024-04-25T11:39:59.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.478+0000 7f5b65f87640 1 -- start start 2024-04-25T11:39:59.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.478+0000 7f5b65f87640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 0x7f5b600cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.478+0000 7f5b65f87640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5b600c7b70 0x7f5b600cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.478+0000 7f5b65f87640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b600ccd00 0x7f5b600cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.478+0000 7f5b65f87640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5b600cc550 con 0x7f5b600ccd00 2024-04-25T11:39:59.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.478+0000 7f5b65f87640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f5b600cd6a0 con 0x7f5b600c7b70 2024-04-25T11:39:59.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.478+0000 7f5b65f87640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6015ae60 con 0x7f5b600cb240 2024-04-25T11:39:59.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b600ccd00 0x7f5b600cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b600ccd00 0x7f5b600cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46800/0 (socket says 172.21.15.40:46800) 2024-04-25T11:39:59.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5ffff640 1 -- 172.21.15.40:0/4244489347 learned_addr learned my addr 172.21.15.40:0/4244489347 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:59.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5effd640 1 --2- 172.21.15.40:0/4244489347 >> v2:172.21.15.62:3300/0 conn(0x7f5b600c7b70 0x7f5b600cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5f7fe640 1 --2- 172.21.15.40:0/4244489347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 0x7f5b600cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5ffff640 1 -- 172.21.15.40:0/4244489347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 msgr2=0x7f5b600cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5ffff640 1 --2- 172.21.15.40:0/4244489347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 0x7f5b600cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5ffff640 1 -- 172.21.15.40:0/4244489347 >> v2:172.21.15.62:3300/0 conn(0x7f5b600c7b70 msgr2=0x7f5b600cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5ffff640 1 --2- 172.21.15.40:0/4244489347 >> v2:172.21.15.62:3300/0 conn(0x7f5b600c7b70 0x7f5b600cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5ffff640 1 -- 172.21.15.40:0/4244489347 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6015b140 con 0x7f5b600ccd00 2024-04-25T11:39:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5effd640 1 --2- 172.21.15.40:0/4244489347 >> v2:172.21.15.62:3300/0 conn(0x7f5b600c7b70 0x7f5b600cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.479+0000 7f5b5f7fe640 1 --2- 172.21.15.40:0/4244489347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 0x7f5b600cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b5ffff640 1 --2- 172.21.15.40:0/4244489347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 0x7f5b600cd160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b600cdf70 tx=0x7f5b5802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a1f94dc165b7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b5e7fc640 1 -- 172.21.15.40:0/4244489347 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58036070 con 0x7f5b600ccd00 2024-04-25T11:39:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b5e7fc640 1 -- 172.21.15.40:0/4244489347 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5b58031600 con 0x7f5b600ccd00 2024-04-25T11:39:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b5e7fc640 1 -- 172.21.15.40:0/4244489347 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b580319f0 con 0x7f5b600ccd00 2024-04-25T11:39:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b65f87640 1 -- 172.21.15.40:0/4244489347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 msgr2=0x7f5b600cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b65f87640 1 --2- 172.21.15.40:0/4244489347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 0x7f5b600cd160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b600cdf70 tx=0x7f5b5802a6a0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b65f87640 1 -- 172.21.15.40:0/4244489347 shutdown_connections 2024-04-25T11:39:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b65f87640 1 --2- 172.21.15.40:0/4244489347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 0x7f5b600cd160 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b65f87640 1 --2- 172.21.15.40:0/4244489347 >> v2:172.21.15.62:3300/0 conn(0x7f5b600c7b70 0x7f5b600cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b65f87640 1 --2- 172.21.15.40:0/4244489347 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 0x7f5b600cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b65f87640 1 -- 172.21.15.40:0/4244489347 >> 172.21.15.40:0/4244489347 conn(0x7f5b6006c3d0 msgr2=0x7f5b6006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b65f87640 1 -- 172.21.15.40:0/4244489347 shutdown_connections 2024-04-25T11:39:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.480+0000 7f5b65f87640 1 -- 172.21.15.40:0/4244489347 wait complete. 2024-04-25T11:39:59.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.482+0000 7f5b65f87640 1 Processor -- start 2024-04-25T11:39:59.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.482+0000 7f5b65f87640 1 -- start start 2024-04-25T11:39:59.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.482+0000 7f5b65f87640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b600c7b70 0x7f5b600c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.482+0000 7f5b65f87640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 0x7f5b600c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.482+0000 7f5b65f87640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 0x7f5b60159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.482+0000 7f5b65f87640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6015b9e0 con 0x7f5b600ccd00 2024-04-25T11:39:59.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.482+0000 7f5b65f87640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6015b700 con 0x7f5b600c7b70 2024-04-25T11:39:59.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.482+0000 7f5b65f87640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5b601552b0 con 0x7f5b600cb240 2024-04-25T11:39:59.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 0x7f5b60159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 0x7f5b60159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46806/0 (socket says 172.21.15.40:46806) 2024-04-25T11:39:59.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5ffff640 1 -- 172.21.15.40:0/701164628 learned_addr learned my addr 172.21.15.40:0/701164628 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:59.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5effd640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 0x7f5b600c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5f7fe640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b600c7b70 0x7f5b600c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5ffff640 1 -- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 msgr2=0x7f5b600c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5ffff640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 0x7f5b600c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5ffff640 1 -- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b600c7b70 msgr2=0x7f5b600c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5ffff640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b600c7b70 0x7f5b600c30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5ffff640 1 -- 172.21.15.40:0/701164628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6015aad0 con 0x7f5b600ccd00 2024-04-25T11:39:59.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b5ffff640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 0x7f5b60159080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5b58034040 tx=0x7f5b58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b47fff640 1 -- 172.21.15.40:0/701164628 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58036050 con 0x7f5b600ccd00 2024-04-25T11:39:59.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b47fff640 1 -- 172.21.15.40:0/701164628 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5b58042070 con 0x7f5b600ccd00 2024-04-25T11:39:59.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b601f1260 con 0x7f5b600ccd00 2024-04-25T11:39:59.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b47fff640 1 -- 172.21.15.40:0/701164628 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58009510 con 0x7f5b600ccd00 2024-04-25T11:39:59.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.483+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b601f1500 con 0x7f5b600ccd00 2024-04-25T11:39:59.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.484+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b60068bb0 con 0x7f5b600ccd00 2024-04-25T11:39:59.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.485+0000 7f5b47fff640 1 -- 172.21.15.40:0/701164628 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b58007bf0 con 0x7f5b600ccd00 2024-04-25T11:39:59.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.485+0000 7f5b47fff640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5b4c06c920 0x7f5b4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.485+0000 7f5b47fff640 1 -- 172.21.15.40:0/701164628 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5b580acb80 con 0x7f5b600ccd00 2024-04-25T11:39:59.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.485+0000 7f5b5f7fe640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5b4c06c920 0x7f5b4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.486+0000 7f5b5f7fe640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5b4c06c920 0x7f5b4c06ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5b50006fd0 tx=0x7f5b50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.489+0000 7f5b47fff640 1 -- 172.21.15.40:0/701164628 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b58045360 con 0x7f5b600ccd00 2024-04-25T11:39:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.714+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b600cc930 con 0x7f5b600ccd00 2024-04-25T11:39:59.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.715+0000 7f5b47fff640 1 -- 172.21.15.40:0/701164628 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6904 (secure 0 0 0) 0x7f5b5807aea0 con 0x7f5b600ccd00 2024-04-25T11:39:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:39:59.702 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1073487120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1073487120},{"type":"v1","addr":"172.21.15.40:6837","nonce":1073487120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:38:25.023156+0000","modified":"2024-04-25T11:38:54.056041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4300,"mds_1":4294,"mds_2":4279,"mds_3":4260,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"d","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6838/3333589902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3333589902},{"type":"v1","addr":"172.21.15.62:6838","nonce":3333589902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4260":{"gid":4260,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4104143657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4104143657},{"type":"v1","addr":"172.21.15.40:6835","nonce":4104143657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6836/535060457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":535060457},{"type":"v1","addr":"172.21.15.40:6836","nonce":535060457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4294":{"gid":4294,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6835/3758619338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3758619338},{"type":"v1","addr":"172.21.15.62:6835","nonce":3758619338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/2121129816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6837","nonce":2121129816},{"type":"v1","addr":"172.21.15.62:6839","nonce":2121129816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4241,4260,4279,4294,4300]},"id":1}]} 2024-04-25T11:39:59.703 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-04-25T11:39:59.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5b4c06c920 msgr2=0x7f5b4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5b4c06c920 0x7f5b4c06ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5b50006fd0 tx=0x7f5b50008040 comp rx=0 tx=0).stop 2024-04-25T11:39:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 msgr2=0x7f5b60159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 0x7f5b60159080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5b58034040 tx=0x7f5b58004a40 comp rx=0 tx=0).stop 2024-04-25T11:39:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 shutdown_connections 2024-04-25T11:39:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f5b4c06c920 0x7f5b4c06ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b600ccd00 0x7f5b60159080 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b600cb240 0x7f5b600c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 --2- 172.21.15.40:0/701164628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b600c7b70 0x7f5b600c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 >> 172.21.15.40:0/701164628 conn(0x7f5b6006c3d0 msgr2=0x7f5b60154510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 shutdown_connections 2024-04-25T11:39:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.717+0000 7f5b65f87640 1 -- 172.21.15.40:0/701164628 wait complete. 2024-04-25T11:39:59.716 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.829+0000 7f375119b640 1 Processor -- start 2024-04-25T11:39:59.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.829+0000 7f375119b640 1 -- start start 2024-04-25T11:39:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f375119b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0cb240 0x7f374c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f375119b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374c15ae60 con 0x7f374c0c7b70 2024-04-25T11:39:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f375119b640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374c15b140 con 0x7f374c0ccd00 2024-04-25T11:39:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f375119b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f374c15b420 con 0x7f374c0cb240 2024-04-25T11:39:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f3743fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f374c0c7b70 0x7f374c0cc010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57182/0 (socket says 172.21.15.40:57182) 2024-04-25T11:39:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f3743fff640 1 -- 172.21.15.40:0/1149679190 learned_addr learned my addr 172.21.15.40:0/1149679190 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:39:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374ad9d640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0cb240 0x7f374c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2697060077 0 0) 0x7f374c15ae60 con 0x7f374c0c7b70 2024-04-25T11:39:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3738003520 con 0x7f374c0c7b70 2024-04-25T11:39:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2591211438 0 0) 0x7f374c15b140 con 0x7f374c0ccd00 2024-04-25T11:39:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f374c15ae60 con 0x7f374c0ccd00 2024-04-25T11:39:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2300160616 0 0) 0x7f3738003520 con 0x7f374c0c7b70 2024-04-25T11:39:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0cb240 msgr2=0x7f374c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0cb240 0x7f374c0cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 >> v1:172.21.15.62:6789/0 conn(0x7f374c0ccd00 legacy=0x7f374c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c15b700 con 0x7f374c0c7b70 2024-04-25T11:39:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7f37340017f0 con 0x7f374c0c7b70 2024-04-25T11:39:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 >> v1:172.21.15.40:6789/0 conn(0x7f374c0c7b70 legacy=0x7f374c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3738003b70 0x7f3738003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0cb240 0x7f3738004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f37380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.830+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f374c15b420 con 0x7f3738004d40 2024-04-25T11:39:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f373800ba70 con 0x7f3738003b70 2024-04-25T11:39:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f373800bc20 con 0x7f374c0cb240 2024-04-25T11:39:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f374b59e640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f37380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f374ad9d640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3738003b70 0x7f3738003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f3743fff640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0cb240 0x7f3738004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f374b59e640 1 -- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0cb240 msgr2=0x7f3738004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f374b59e640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0cb240 0x7f3738004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f374b59e640 1 -- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3738003b70 msgr2=0x7f3738003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f374b59e640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3738003b70 0x7f3738003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.831+0000 7f374b59e640 1 -- 172.21.15.40:0/1149679190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3738003520 con 0x7f3738004d40 2024-04-25T11:39:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f374b59e640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f37380051c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3744001fa0 tx=0x7f3738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130a20042f4fa256 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744001bf0 con 0x7f3738004d40 2024-04-25T11:39:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3744003b70 con 0x7f3738004d40 2024-04-25T11:39:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f374a59c640 1 -- 172.21.15.40:0/1149679190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37440050d0 con 0x7f3738004d40 2024-04-25T11:39:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f375119b640 1 -- 172.21.15.40:0/1149679190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 msgr2=0x7f37380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f375119b640 1 --2- 172.21.15.40:0/1149679190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f37380051c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3744001fa0 tx=0x7f3738005830 comp rx=0 tx=0).stop 2024-04-25T11:39:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f374b59e640 1 -- 172.21.15.40:0/1149679190 reap_dead start 2024-04-25T11:39:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f375119b640 1 -- 172.21.15.40:0/1149679190 shutdown_connections 2024-04-25T11:39:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f375119b640 1 -- 172.21.15.40:0/1149679190 >> 172.21.15.40:0/1149679190 conn(0x7f374c06c3d0 msgr2=0x7f374c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f375119b640 1 -- 172.21.15.40:0/1149679190 shutdown_connections 2024-04-25T11:39:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.832+0000 7f375119b640 1 -- 172.21.15.40:0/1149679190 wait complete. 2024-04-25T11:39:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.834+0000 7f375119b640 1 Processor -- start 2024-04-25T11:39:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.834+0000 7f375119b640 1 -- start start 2024-04-25T11:39:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.834+0000 7f375119b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f374c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.834+0000 7f375119b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0c88f0 0x7f374c150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.834+0000 7f375119b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f374c15a820 0x7f374c0c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.834+0000 7f375119b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1552b0 con 0x7f3738004d40 2024-04-25T11:39:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.834+0000 7f375119b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f374c0b6290 con 0x7f374c15a820 2024-04-25T11:39:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.834+0000 7f375119b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f374c0c8460 con 0x7f374c0c88f0 2024-04-25T11:39:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f374c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f374c151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46820/0 (socket says 172.21.15.40:46820) 2024-04-25T11:39:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374ad9d640 1 -- 172.21.15.40:0/1055783642 learned_addr learned my addr 172.21.15.40:0/1055783642 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:39:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374ad9d640 1 -- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0c88f0 msgr2=0x7f374c150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f3743fff640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0c88f0 0x7f374c150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374b59e640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f374c15a820 0x7f374c0c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374ad9d640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0c88f0 0x7f374c150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374ad9d640 1 -- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f374c15a820 msgr2=0x7f374c0c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374ad9d640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f374c15a820 0x7f374c0c4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374ad9d640 1 -- 172.21.15.40:0/1055783642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c152160 con 0x7f3738004d40 2024-04-25T11:39:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f374ad9d640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f374c151c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f373c0119d0 tx=0x7f373c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f37437fe640 1 -- 172.21.15.40:0/1055783642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c00fa90 con 0x7f3738004d40 2024-04-25T11:39:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f37437fe640 1 -- 172.21.15.40:0/1055783642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f373c00fc30 con 0x7f3738004d40 2024-04-25T11:39:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f37437fe640 1 -- 172.21.15.40:0/1055783642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c014440 con 0x7f3738004d40 2024-04-25T11:39:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c0cec40 con 0x7f3738004d40 2024-04-25T11:39:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.835+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c0c80f0 con 0x7f3738004d40 2024-04-25T11:39:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.836+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374c068bb0 con 0x7f3738004d40 2024-04-25T11:39:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.837+0000 7f37437fe640 1 -- 172.21.15.40:0/1055783642 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f373c012040 con 0x7f3738004d40 2024-04-25T11:39:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.837+0000 7f37437fe640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f372c06c8f0 0x7f372c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.837+0000 7f37437fe640 1 -- 172.21.15.40:0/1055783642 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f373c08c290 con 0x7f3738004d40 2024-04-25T11:39:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.837+0000 7f3743fff640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f372c06c8f0 0x7f372c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.838+0000 7f3743fff640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f372c06c8f0 0x7f372c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f37340011d0 tx=0x7f3734009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:39:59.842+0000 7f37437fe640 1 -- 172.21.15.40:0/1055783642 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f373c019030 con 0x7f3738004d40 2024-04-25T11:40:00.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.071+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f374c0cbdf0 con 0x7f3738004d40 2024-04-25T11:40:00.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.071+0000 7f37437fe640 1 -- 172.21.15.40:0/1055783642 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6904 (secure 0 0 0) 0x7f373c05a5b0 con 0x7f3738004d40 2024-04-25T11:40:00.058 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:40:00.059 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/1073487120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1073487120},{"type":"v1","addr":"172.21.15.40:6837","nonce":1073487120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:38:25.023156+0000","modified":"2024-04-25T11:38:54.056041+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4300,"mds_1":4294,"mds_2":4279,"mds_3":4260,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"d","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.62:6838/3333589902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3333589902},{"type":"v1","addr":"172.21.15.62:6838","nonce":3333589902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4260":{"gid":4260,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.40:6835/4104143657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4104143657},{"type":"v1","addr":"172.21.15.40:6835","nonce":4104143657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6836/535060457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":535060457},{"type":"v1","addr":"172.21.15.40:6836","nonce":535060457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4294":{"gid":4294,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.62:6835/3758619338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3758619338},{"type":"v1","addr":"172.21.15.62:6835","nonce":3758619338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6839/2121129816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6837","nonce":2121129816},{"type":"v1","addr":"172.21.15.62:6839","nonce":2121129816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4241,4260,4279,4294,4300]},"id":1}]} 2024-04-25T11:40:00.059 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-04-25T11:40:00.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f372c06c8f0 msgr2=0x7f372c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f372c06c8f0 0x7f372c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f37340011d0 tx=0x7f3734009040 comp rx=0 tx=0).stop 2024-04-25T11:40:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 msgr2=0x7f374c151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f374c151c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f373c0119d0 tx=0x7f373c011e90 comp rx=0 tx=0).stop 2024-04-25T11:40:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 shutdown_connections 2024-04-25T11:40:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f372c06c8f0 0x7f372c06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f374c15a820 0x7f374c0c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f374c0c88f0 0x7f374c150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 --2- 172.21.15.40:0/1055783642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3738004d40 0x7f374c151c20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.073+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 >> 172.21.15.40:0/1055783642 conn(0x7f374c06c3d0 msgr2=0x7f374c06c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:00.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.074+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 shutdown_connections 2024-04-25T11:40:00.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.074+0000 7f375119b640 1 -- 172.21.15.40:0/1055783642 wait complete. 2024-04-25T11:40:00.071 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:40:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.188+0000 7f3c33722640 1 Processor -- start 2024-04-25T11:40:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.188+0000 7f3c33722640 1 -- start start 2024-04-25T11:40:00.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.189+0000 7f3c33722640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c2c0b6210 0x7f3c2c0c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.189+0000 7f3c33722640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c15f060 con 0x7f3c2c0c8690 2024-04-25T11:40:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.189+0000 7f3c33722640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c0d1040 con 0x7f3c2c154f50 2024-04-25T11:40:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.189+0000 7f3c33722640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3c2c15eb20 con 0x7f3c2c0b6210 2024-04-25T11:40:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.189+0000 7f3c30cbd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c2c0b6210 0x7f3c2c0c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.189+0000 7f3c31cbf640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3c2c0c8690 0x7f3c2c0c8ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57192/0 (socket says 172.21.15.40:57192) 2024-04-25T11:40:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.189+0000 7f3c31cbf640 1 -- 172.21.15.40:0/670034364 learned_addr learned my addr 172.21.15.40:0/670034364 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:40:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.190+0000 7f3c30cbd640 1 -- 172.21.15.40:0/670034364 >> v1:172.21.15.62:6789/0 conn(0x7f3c2c154f50 legacy=0x7f3c2c0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.190+0000 7f3c30cbd640 1 -- 172.21.15.40:0/670034364 >> v1:172.21.15.40:6789/0 conn(0x7f3c2c0c8690 legacy=0x7f3c2c0c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.190+0000 7f3c30cbd640 1 -- 172.21.15.40:0/670034364 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c0cc150 con 0x7f3c2c0b6210 2024-04-25T11:40:00.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.190+0000 7f3c30cbd640 1 --2- 172.21.15.40:0/670034364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0b6210 0x7f3c2c0c7a70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0cc320 tx=0x7f3c1c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6aedb3c48f5400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.190+0000 7f3c23fff640 1 -- 172.21.15.40:0/670034364 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c037070 con 0x7f3c2c0b6210 2024-04-25T11:40:00.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.190+0000 7f3c23fff640 1 -- 172.21.15.40:0/670034364 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3c1c032440 con 0x7f3c2c0b6210 2024-04-25T11:40:00.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.191+0000 7f3c23fff640 1 -- 172.21.15.40:0/670034364 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c032830 con 0x7f3c2c0b6210 2024-04-25T11:40:00.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.191+0000 7f3c33722640 1 -- 172.21.15.40:0/670034364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0b6210 msgr2=0x7f3c2c0c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.191+0000 7f3c33722640 1 --2- 172.21.15.40:0/670034364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0b6210 0x7f3c2c0c7a70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0cc320 tx=0x7f3c1c02a180 comp rx=0 tx=0).stop 2024-04-25T11:40:00.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.191+0000 7f3c33722640 1 -- 172.21.15.40:0/670034364 shutdown_connections 2024-04-25T11:40:00.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.191+0000 7f3c33722640 1 --2- 172.21.15.40:0/670034364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0b6210 0x7f3c2c0c7a70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.191+0000 7f3c33722640 1 -- 172.21.15.40:0/670034364 >> 172.21.15.40:0/670034364 conn(0x7f3c2c0d0440 msgr2=0x7f3c2c0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:00.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.191+0000 7f3c33722640 1 -- 172.21.15.40:0/670034364 shutdown_connections 2024-04-25T11:40:00.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.191+0000 7f3c33722640 1 -- 172.21.15.40:0/670034364 wait complete. 2024-04-25T11:40:00.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.193+0000 7f3c33722640 1 Processor -- start 2024-04-25T11:40:00.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.193+0000 7f3c33722640 1 -- start start 2024-04-25T11:40:00.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.193+0000 7f3c33722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0c8690 0x7f3c2c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.193+0000 7f3c33722640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2c154f50 0x7f3c2c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.193+0000 7f3c33722640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c2c1f4030 0x7f3c2c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.193+0000 7f3c33722640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c0cc3f0 con 0x7f3c2c0c8690 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.193+0000 7f3c33722640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c15ed80 con 0x7f3c2c154f50 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.193+0000 7f3c33722640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c0c8170 con 0x7f3c2c1f4030 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c314be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0c8690 0x7f3c2c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c314be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0c8690 0x7f3c2c06c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46842/0 (socket says 172.21.15.40:46842) 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c314be640 1 -- 172.21.15.40:0/387203398 learned_addr learned my addr 172.21.15.40:0/387203398 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c31cbf640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c2c1f4030 0x7f3c2c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c30cbd640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2c154f50 0x7f3c2c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c314be640 1 -- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c2c1f4030 msgr2=0x7f3c2c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c314be640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c2c1f4030 0x7f3c2c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c314be640 1 -- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2c154f50 msgr2=0x7f3c2c15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c314be640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2c154f50 0x7f3c2c15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c314be640 1 -- 172.21.15.40:0/387203398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c1f0dd0 con 0x7f3c2c0c8690 2024-04-25T11:40:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c30cbd640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2c154f50 0x7f3c2c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c31cbf640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c2c1f4030 0x7f3c2c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c314be640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0c8690 0x7f3c2c06c370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800c400 tx=0x7f3c2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c21ffb640 1 -- 172.21.15.40:0/387203398 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c280052d0 con 0x7f3c2c0c8690 2024-04-25T11:40:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c21ffb640 1 -- 172.21.15.40:0/387203398 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3c28005490 con 0x7f3c2c0c8690 2024-04-25T11:40:00.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c1f10b0 con 0x7f3c2c0c8690 2024-04-25T11:40:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.194+0000 7f3c21ffb640 1 -- 172.21.15.40:0/387203398 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28007280 con 0x7f3c2c0c8690 2024-04-25T11:40:00.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.195+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c1f1670 con 0x7f3c2c0c8690 2024-04-25T11:40:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.196+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c068bb0 con 0x7f3c2c0c8690 2024-04-25T11:40:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.196+0000 7f3c21ffb640 1 -- 172.21.15.40:0/387203398 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c28007420 con 0x7f3c2c0c8690 2024-04-25T11:40:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.196+0000 7f3c21ffb640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3c1006c920 0x7f3c1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.197+0000 7f3c21ffb640 1 -- 172.21.15.40:0/387203398 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3c2808b130 con 0x7f3c2c0c8690 2024-04-25T11:40:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.197+0000 7f3c30cbd640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3c1006c920 0x7f3c1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.197+0000 7f3c30cbd640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3c1006c920 0x7f3c1006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c034000 tx=0x7f3c1c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.201+0000 7f3c21ffb640 1 -- 172.21.15.40:0/387203398 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c280594e0 con 0x7f3c2c0c8690 2024-04-25T11:40:00.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.388+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c2c0cbe00 con 0x7f3c2c0c8690 2024-04-25T11:40:00.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.389+0000 7f3c21ffb640 1 -- 172.21.15.40:0/387203398 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16806 (secure 0 0 0) 0x7f3c28058de0 con 0x7f3c2c0c8690 2024-04-25T11:40:00.376 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T11:40:00.376 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"6d14dde7-764d-42d0-be37-c790a704858d","created":"2024-04-25T11:38:08.558886+0000","modified":"2024-04-25T11:39:14.106471+0000","last_up_change":"2024-04-25T11:38:18.548496+0000","last_in_change":"2024-04-25T11:38:10.819113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:38:18.972986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:38:23.454183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:38:23.968333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f522bed8-ab40-43ae-8616-996ef5e1732c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6817","nonce":2617970798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6821","nonce":2617970798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6828","nonce":2617970798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":2617970798},{"type":"v1","addr":"172.21.15.40:6825","nonce":2617970798}]},"public_addr":"172.21.15.40:6817/2617970798","cluster_addr":"172.21.15.40:6821/2617970798","heartbeat_back_addr":"172.21.15.40:6828/2617970798","heartbeat_front_addr":"172.21.15.40:6825/2617970798","state":["exists","up"]},{"osd":1,"uuid":"713a8175-952b-4888-90e2-9da6f79016a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6801","nonce":3790556570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6803","nonce":3790556570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6807","nonce":3790556570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3790556570},{"type":"v1","addr":"172.21.15.40:6805","nonce":3790556570}]},"public_addr":"172.21.15.40:6801/3790556570","cluster_addr":"172.21.15.40:6803/3790556570","heartbeat_back_addr":"172.21.15.40:6807/3790556570","heartbeat_front_addr":"172.21.15.40:6805/3790556570","state":["exists","up"]},{"osd":2,"uuid":"789f733e-f414-4093-9ad1-444b395333a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6809","nonce":2321494678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6811","nonce":2321494678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6815","nonce":2321494678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2321494678},{"type":"v1","addr":"172.21.15.40:6813","nonce":2321494678}]},"public_addr":"172.21.15.40:6809/2321494678","cluster_addr":"172.21.15.40:6811/2321494678","heartbeat_back_addr":"172.21.15.40:6815/2321494678","heartbeat_front_addr":"172.21.15.40:6813/2321494678","state":["exists","up"]},{"osd":3,"uuid":"41202350-9bfc-49ad-90f4-59c63e314ea4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6820","nonce":364049698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6824","nonce":364049698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6831","nonce":364049698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":364049698},{"type":"v1","addr":"172.21.15.40:6829","nonce":364049698}]},"public_addr":"172.21.15.40:6820/364049698","cluster_addr":"172.21.15.40:6824/364049698","heartbeat_back_addr":"172.21.15.40:6831/364049698","heartbeat_front_addr":"172.21.15.40:6829/364049698","state":["exists","up"]},{"osd":4,"uuid":"ba3ad959-4d4a-48ab-8e7d-a2a9151280ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6801","nonce":2239863089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6803","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6805","nonce":2239863089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6811","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6813","nonce":2239863089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6807","nonce":2239863089},{"type":"v1","addr":"172.21.15.62:6809","nonce":2239863089}]},"public_addr":"172.21.15.62:6801/2239863089","cluster_addr":"172.21.15.62:6805/2239863089","heartbeat_back_addr":"172.21.15.62:6813/2239863089","heartbeat_front_addr":"172.21.15.62:6809/2239863089","state":["exists","up"]},{"osd":5,"uuid":"bd3dbba0-ba11-4cd2-b930-d884380986b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6825","nonce":3210426647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6827","nonce":3210426647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6831","nonce":3210426647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3210426647},{"type":"v1","addr":"172.21.15.62:6829","nonce":3210426647}]},"public_addr":"172.21.15.62:6825/3210426647","cluster_addr":"172.21.15.62:6827/3210426647","heartbeat_back_addr":"172.21.15.62:6831/3210426647","heartbeat_front_addr":"172.21.15.62:6829/3210426647","state":["exists","up"]},{"osd":6,"uuid":"c97e06d6-a79a-4ff1-a5ae-404ae70f8638","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6804","nonce":1111875962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6808","nonce":1111875962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6815","nonce":1111875962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1111875962},{"type":"v1","addr":"172.21.15.62:6812","nonce":1111875962}]},"public_addr":"172.21.15.62:6804/1111875962","cluster_addr":"172.21.15.62:6808/1111875962","heartbeat_back_addr":"172.21.15.62:6815/1111875962","heartbeat_front_addr":"172.21.15.62:6812/1111875962","state":["exists","up"]},{"osd":7,"uuid":"83214518-6405-4702-8877-900f7b544f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6817","nonce":3160032582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6819","nonce":3160032582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6823","nonce":3160032582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3160032582},{"type":"v1","addr":"172.21.15.62:6821","nonce":3160032582}]},"public_addr":"172.21.15.62:6817/3160032582","cluster_addr":"172.21.15.62:6819/3160032582","heartbeat_back_addr":"172.21.15.62:6823/3160032582","heartbeat_front_addr":"172.21.15.62:6821/3160032582","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.951241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.956245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.966736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.007766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.958329+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.000133+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:16.004671+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:15.939387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:40:00.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3c1006c920 msgr2=0x7f3c1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3c1006c920 0x7f3c1006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c034000 tx=0x7f3c1c02e000 comp rx=0 tx=0).stop 2024-04-25T11:40:00.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0c8690 msgr2=0x7f3c2c06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0c8690 0x7f3c2c06c370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800c400 tx=0x7f3c2800c8c0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 shutdown_connections 2024-04-25T11:40:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f3c1006c920 0x7f3c1006ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c2c1f4030 0x7f3c2c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2c154f50 0x7f3c2c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 --2- 172.21.15.40:0/387203398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c2c0c8690 0x7f3c2c06c370 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 >> 172.21.15.40:0/387203398 conn(0x7f3c2c0d0440 msgr2=0x7f3c2c1506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 shutdown_connections 2024-04-25T11:40:00.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.391+0000 7f3c33722640 1 -- 172.21.15.40:0/387203398 wait complete. 2024-04-25T11:40:00.389 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:40:00.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.503+0000 7fbbb7ec2640 1 Processor -- start 2024-04-25T11:40:00.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.503+0000 7fbbb7ec2640 1 -- start start 2024-04-25T11:40:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.504+0000 7fbbb7ec2640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbbb00cb240 0x7fbbb00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.504+0000 7fbbb7ec2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb015ae60 con 0x7fbbb00ccd00 2024-04-25T11:40:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.504+0000 7fbbb7ec2640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb015b140 con 0x7fbbb00c7b70 2024-04-25T11:40:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.504+0000 7fbbb7ec2640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb015b420 con 0x7fbbb00cb240 2024-04-25T11:40:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.504+0000 7fbbb645f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbbb00ccd00 0x7fbbb00cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57198/0 (socket says 172.21.15.40:57198) 2024-04-25T11:40:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.504+0000 7fbbb645f640 1 -- 172.21.15.40:0/3511484145 learned_addr learned my addr 172.21.15.40:0/3511484145 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-25T11:40:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.504+0000 7fbbb5c5e640 1 --2- 172.21.15.40:0/3511484145 >> v2:172.21.15.62:3300/0 conn(0x7fbbb00cb240 0x7fbbb00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3951387436 0 0) 0x7fbbb015ae60 con 0x7fbbb00ccd00 2024-04-25T11:40:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb98003520 con 0x7fbbb00ccd00 2024-04-25T11:40:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 445358105 0 0) 0x7fbb98003520 con 0x7fbbb00ccd00 2024-04-25T11:40:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 >> v2:172.21.15.62:3300/0 conn(0x7fbbb00cb240 msgr2=0x7fbbb00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 --2- 172.21.15.40:0/3511484145 >> v2:172.21.15.62:3300/0 conn(0x7fbbb00cb240 0x7fbbb00cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 >> v1:172.21.15.62:6789/0 conn(0x7fbbb00c7b70 legacy=0x7fbbb00cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb015b700 con 0x7fbbb00ccd00 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1214400544 0 0) 0x7fbba8001670 con 0x7fbbb00ccd00 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb5c5e640 1 --2- 172.21.15.40:0/3511484145 >> v2:172.21.15.62:3300/0 conn(0x7fbbb00cb240 0x7fbbb00cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 >> v1:172.21.15.40:6789/0 conn(0x7fbbb00ccd00 legacy=0x7fbbb00cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbb98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb98004800 0x7fbb98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980051c0 0x7fbb9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb015b420 con 0x7fbb980039b0 2024-04-25T11:40:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9800c060 con 0x7fbb980051c0 2024-04-25T11:40:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9800c260 con 0x7fbb98004800 2024-04-25T11:40:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb5c5e640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbb98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb645f640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980051c0 0x7fbb9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb545d640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb98004800 0x7fbb98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb5c5e640 1 -- 172.21.15.40:0/3511484145 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb98004800 msgr2=0x7fbb98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb5c5e640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb98004800 0x7fbb98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb5c5e640 1 -- 172.21.15.40:0/3511484145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980051c0 msgr2=0x7fbb9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb5c5e640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980051c0 0x7fbb9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb5c5e640 1 -- 172.21.15.40:0/3511484145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb98003520 con 0x7fbb980039b0 2024-04-25T11:40:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb645f640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb980051c0 0x7fbb9800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:40:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.505+0000 7fbbb645f640 1 -- 172.21.15.40:0/3511484145 reap_dead start 2024-04-25T11:40:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.506+0000 7fbbb5c5e640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbb98003dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e350 tx=0x7fbb98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c3334075a78048 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.506+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0033070 con 0x7fbb980039b0 2024-04-25T11:40:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.506+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbba002f660 con 0x7fbb980039b0 2024-04-25T11:40:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.506+0000 7fbbb4c5c640 1 -- 172.21.15.40:0/3511484145 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0009de0 con 0x7fbb980039b0 2024-04-25T11:40:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.507+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3511484145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 msgr2=0x7fbb98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.507+0000 7fbbb7ec2640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbb98003dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e350 tx=0x7fbb98005830 comp rx=0 tx=0).stop 2024-04-25T11:40:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.507+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3511484145 shutdown_connections 2024-04-25T11:40:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.507+0000 7fbbb7ec2640 1 --2- 172.21.15.40:0/3511484145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbb98003dd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.507+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3511484145 >> 172.21.15.40:0/3511484145 conn(0x7fbbb006c3d0 msgr2=0x7fbbb006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.507+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3511484145 shutdown_connections 2024-04-25T11:40:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.507+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3511484145 wait complete. 2024-04-25T11:40:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.508+0000 7fbbb7ec2640 1 Processor -- start 2024-04-25T11:40:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.508+0000 7fbbb7ec2640 1 -- start start 2024-04-25T11:40:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb7ec2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbbb00c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb7ec2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb0150650 0x7fbbb00c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb7ec2640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb01f1390 0x7fbbb01f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb7ec2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01552b0 con 0x7fbb980039b0 2024-04-25T11:40:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb7ec2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb00b6290 con 0x7fbbb0150650 2024-04-25T11:40:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb7ec2640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb00c6d10 con 0x7fbbb01f1390 2024-04-25T11:40:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb5c5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbbb00c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb5c5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbbb00c4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46862/0 (socket says 172.21.15.40:46862) 2024-04-25T11:40:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb5c5e640 1 -- 172.21.15.40:0/3909731662 learned_addr learned my addr 172.21.15.40:0/3909731662 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:40:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb645f640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb01f1390 0x7fbbb01f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb545d640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb0150650 0x7fbbb00c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb5c5e640 1 -- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb01f1390 msgr2=0x7fbbb01f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb5c5e640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb01f1390 0x7fbbb01f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb5c5e640 1 -- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb0150650 msgr2=0x7fbbb00c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb5c5e640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb0150650 0x7fbbb00c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb5c5e640 1 -- 172.21.15.40:0/3909731662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb01f0eb0 con 0x7fbb980039b0 2024-04-25T11:40:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbb5c5e640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbbb00c4050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e440 tx=0x7fbba0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.509+0000 7fbbae7fc640 1 -- 172.21.15.40:0/3909731662 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0033040 con 0x7fbb980039b0 2024-04-25T11:40:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.510+0000 7fbbae7fc640 1 -- 172.21.15.40:0/3909731662 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbba002f460 con 0x7fbb980039b0 2024-04-25T11:40:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.510+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb00c6540 con 0x7fbb980039b0 2024-04-25T11:40:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.510+0000 7fbbae7fc640 1 -- 172.21.15.40:0/3909731662 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0038c80 con 0x7fbb980039b0 2024-04-25T11:40:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.510+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb01f51a0 con 0x7fbb980039b0 2024-04-25T11:40:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.510+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb0068bb0 con 0x7fbb980039b0 2024-04-25T11:40:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.511+0000 7fbbae7fc640 1 -- 172.21.15.40:0/3909731662 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbba0036070 con 0x7fbb980039b0 2024-04-25T11:40:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.511+0000 7fbbae7fc640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbb9c06c8f0 0x7fbb9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.511+0000 7fbbae7fc640 1 -- 172.21.15.40:0/3909731662 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbba00ac960 con 0x7fbb980039b0 2024-04-25T11:40:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.512+0000 7fbbb545d640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbb9c06c8f0 0x7fbb9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.512+0000 7fbbb545d640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbb9c06c8f0 0x7fbb9c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0150410 tx=0x7fbba4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.516+0000 7fbbae7fc640 1 -- 172.21.15.40:0/3909731662 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbba0038e20 con 0x7fbb980039b0 2024-04-25T11:40:00.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.741+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbbb00cbdf0 con 0x7fbb980039b0 2024-04-25T11:40:00.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.743+0000 7fbbae7fc640 1 -- 172.21.15.40:0/3909731662 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fbba007ac80 con 0x7fbb980039b0 2024-04-25T11:40:00.730 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:40:00.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbb9c06c8f0 msgr2=0x7fbb9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbb9c06c8f0 0x7fbb9c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0150410 tx=0x7fbba4009040 comp rx=0 tx=0).stop 2024-04-25T11:40:00.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 msgr2=0x7fbbb00c4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbbb00c4050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e440 tx=0x7fbba0009750 comp rx=0 tx=0).stop 2024-04-25T11:40:00.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 shutdown_connections 2024-04-25T11:40:00.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fbb9c06c8f0 0x7fbb9c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbb01f1390 0x7fbbb01f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbb0150650 0x7fbbb00c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 --2- 172.21.15.40:0/3909731662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980039b0 0x7fbbb00c4050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 >> 172.21.15.40:0/3909731662 conn(0x7fbbb006c3d0 msgr2=0x7fbbb0153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:00.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 shutdown_connections 2024-04-25T11:40:00.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.747+0000 7fbbb7ec2640 1 -- 172.21.15.40:0/3909731662 wait complete. 2024-04-25T11:40:00.740 INFO:tasks.ceph.mds.b.smithi062.stderr: -4000> 2024-04-25T11:38:40.267+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:40:00.741 INFO:tasks.ceph.mds.b.smithi062.stderr: -3998> 2024-04-25T11:38:40.267+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T11:40:00.741 INFO:tasks.ceph.mds.b.smithi062.stderr: -3721> 2024-04-25T11:38:45.267+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T11:40:00.742 INFO:tasks.ceph.mds.b.smithi062.stderr: -3402> 2024-04-25T11:38:50.266+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T11:40:00.742 INFO:tasks.ceph.mds.b.smithi062.stderr: -3093> 2024-04-25T11:38:55.266+0000 7f20ab371640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T11:40:00.746 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:40:00.763 INFO:tasks.ceph.mds.e.smithi040.stdout:starting mds.e at 2024-04-25T11:40:00.763 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-25T11:40:00.768 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-04-25T11:40:00.772 INFO:tasks.ceph.mds.f.smithi062.stdout:starting mds.f at 2024-04-25T11:40:00.776 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-04-25T11:40:00.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.864+0000 7f48dfe6b640 1 Processor -- start 2024-04-25T11:40:00.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.864+0000 7f48dfe6b640 1 -- start start 2024-04-25T11:40:00.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48dfe6b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48d80ce750 0x7f48d8070850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48dfe6b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 0x7f48d8157e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48dfe6b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d80ca9c0 con 0x7f48d80c5330 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48dfe6b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f48d80cac60 con 0x7f48d80ce750 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48dfe6b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f48d80703c0 con 0x7f48d8070d90 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48ddc07640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48d80ce750 0x7f48d8070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48ddc07640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48d80ce750 0x7f48d8070850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46894/0 (socket says 172.21.15.40:46894) 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48de408640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f48d80c5330 0x7f48d8158340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57210/0 (socket says 172.21.15.40:57210) 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48ddc07640 1 -- 172.21.15.40:0/2318077147 learned_addr learned my addr 172.21.15.40:0/2318077147 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.865+0000 7f48dd406640 1 --2- 172.21.15.40:0/2318077147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 0x7f48d8157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.866+0000 7f48ddc07640 1 -- 172.21.15.40:0/2318077147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 msgr2=0x7f48d8157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:00.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.866+0000 7f48ddc07640 1 --2- 172.21.15.40:0/2318077147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 0x7f48d8157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.866+0000 7f48ddc07640 1 -- 172.21.15.40:0/2318077147 >> v1:172.21.15.40:6789/0 conn(0x7f48d80c5330 legacy=0x7f48d8158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.866+0000 7f48ddc07640 1 -- 172.21.15.40:0/2318077147 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d80c9760 con 0x7f48d80ce750 2024-04-25T11:40:00.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.866+0000 7f48dcc05640 1 -- 172.21.15.40:0/2318077147 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867461899 0 0) 0x7f48d80ca9c0 con 0x7f48d80c5330 2024-04-25T11:40:00.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.866+0000 7f48ddc07640 1 --2- 172.21.15.40:0/2318077147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80ce750 0x7f48d8070850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009870 tx=0x7f48c802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f17d431ae794e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.866+0000 7f48dcc05640 1 -- 172.21.15.40:0/2318077147 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8037070 con 0x7f48d80ce750 2024-04-25T11:40:00.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.866+0000 7f48dcc05640 1 -- 172.21.15.40:0/2318077147 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f48c802ae50 con 0x7f48d80ce750 2024-04-25T11:40:00.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.866+0000 7f48dcc05640 1 -- 172.21.15.40:0/2318077147 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8032880 con 0x7f48d80ce750 2024-04-25T11:40:00.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.867+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2318077147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80ce750 msgr2=0x7f48d8070850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.867+0000 7f48dfe6b640 1 --2- 172.21.15.40:0/2318077147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80ce750 0x7f48d8070850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009870 tx=0x7f48c802a300 comp rx=0 tx=0).stop 2024-04-25T11:40:00.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.867+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2318077147 shutdown_connections 2024-04-25T11:40:00.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.867+0000 7f48dfe6b640 1 --2- 172.21.15.40:0/2318077147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 0x7f48d8157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.867+0000 7f48dfe6b640 1 --2- 172.21.15.40:0/2318077147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80ce750 0x7f48d8070850 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.867+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2318077147 >> 172.21.15.40:0/2318077147 conn(0x7f48d80676b0 msgr2=0x7f48d8067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:00.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.867+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2318077147 shutdown_connections 2024-04-25T11:40:00.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.867+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2318077147 wait complete. 2024-04-25T11:40:00.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.868+0000 7f48dfe6b640 1 Processor -- start 2024-04-25T11:40:00.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.868+0000 7f48dfe6b640 1 -- start start 2024-04-25T11:40:00.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dfe6b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 0x7f48d8150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dfe6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80c5330 0x7f48d81f03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dfe6b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d80ce750 0x7f48d81f0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dfe6b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f48d80ca000 con 0x7f48d80c5330 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dfe6b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48d80c9d20 con 0x7f48d80ce750 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dfe6b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f48d815e390 con 0x7f48d8070d90 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dd406640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80c5330 0x7f48d81f03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dd406640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80c5330 0x7f48d81f03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46896/0 (socket says 172.21.15.40:46896) 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dd406640 1 -- 172.21.15.40:0/2047419272 learned_addr learned my addr 172.21.15.40:0/2047419272 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48ddc07640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 0x7f48d8150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48de408640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d80ce750 0x7f48d81f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dd406640 1 -- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 msgr2=0x7f48d8150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dd406640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 0x7f48d8150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dd406640 1 -- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d80ce750 msgr2=0x7f48d81f0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dd406640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d80ce750 0x7f48d81f0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.869+0000 7f48dd406640 1 -- 172.21.15.40:0/2047419272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d81f44e0 con 0x7f48d80c5330 2024-04-25T11:40:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.870+0000 7f48dd406640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80c5330 0x7f48d81f03e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e9b0 tx=0x7f48d400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.870+0000 7f48ce7fc640 1 -- 172.21.15.40:0/2047419272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4017070 con 0x7f48d80c5330 2024-04-25T11:40:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.870+0000 7f48ce7fc640 1 -- 172.21.15.40:0/2047419272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f48d400c9a0 con 0x7f48d80c5330 2024-04-25T11:40:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.870+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d80cc7f0 con 0x7f48d80c5330 2024-04-25T11:40:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.870+0000 7f48ce7fc640 1 -- 172.21.15.40:0/2047419272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d400cd90 con 0x7f48d80c5330 2024-04-25T11:40:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.870+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d80ccdb0 con 0x7f48d80c5330 2024-04-25T11:40:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.871+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d8068bb0 con 0x7f48d80c5330 2024-04-25T11:40:00.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.871+0000 7f48ce7fc640 1 -- 172.21.15.40:0/2047419272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48d401a070 con 0x7f48d80c5330 2024-04-25T11:40:00.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.872+0000 7f48ce7fc640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48bc06c920 0x7f48bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.872+0000 7f48ce7fc640 1 -- 172.21.15.40:0/2047419272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6255+0+0 (secure 0 0 0) 0x7f48d408c0d0 con 0x7f48d80c5330 2024-04-25T11:40:00.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.872+0000 7f48ddc07640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48bc06c920 0x7f48bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.872+0000 7f48ddc07640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48bc06c920 0x7f48bc06ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f48c8034000 tx=0x7f48c802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:00.876+0000 7f48ce7fc640 1 -- 172.21.15.40:0/2047419272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48d4059c00 con 0x7f48d80c5330 2024-04-25T11:40:01.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.101+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f48d806fbe0 con 0x7f48d80c5330 2024-04-25T11:40:01.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.748+0000 7f48ce7fc640 1 -- 172.21.15.40:0/2047419272 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f48d405df00 con 0x7f48d80c5330 2024-04-25T11:40:01.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48bc06c920 msgr2=0x7f48bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48bc06c920 0x7f48bc06ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f48c8034000 tx=0x7f48c802e000 comp rx=0 tx=0).stop 2024-04-25T11:40:01.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80c5330 msgr2=0x7f48d81f03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80c5330 0x7f48d81f03e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e9b0 tx=0x7f48d400ee70 comp rx=0 tx=0).stop 2024-04-25T11:40:01.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 shutdown_connections 2024-04-25T11:40:01.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7f48bc06c920 0x7f48bc06ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48d80ce750 0x7f48d81f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48d80c5330 0x7f48d81f03e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 --2- 172.21.15.40:0/2047419272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48d8070d90 0x7f48d8150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 >> 172.21.15.40:0/2047419272 conn(0x7f48d80676b0 msgr2=0x7f48d8067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 shutdown_connections 2024-04-25T11:40:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.752+0000 7f48dfe6b640 1 -- 172.21.15.40:0/2047419272 wait complete. 2024-04-25T11:40:01.753 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:40:01.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.920+0000 7fa66a79b640 1 Processor -- start 2024-04-25T11:40:01.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.920+0000 7fa66a79b640 1 -- start start 2024-04-25T11:40:01.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa66a79b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa664155140 0x7fa6640cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa66a79b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6640c5db0 0x7fa66415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa66a79b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 0x7fa6640d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa66a79b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa6640c5440 con 0x7fa6640c5db0 2024-04-25T11:40:01.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa66a79b640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa6640c5690 con 0x7fa664155140 2024-04-25T11:40:01.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa66a79b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa6640c5970 con 0x7fa66415eeb0 2024-04-25T11:40:01.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa6637fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6640c5db0 0x7fa66415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa6637fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa6640c5db0 0x7fa66415b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46902/0 (socket says 172.21.15.40:46902) 2024-04-25T11:40:01.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa6637fe640 1 -- 172.21.15.40:0/3450799955 learned_addr learned my addr 172.21.15.40:0/3450799955 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:40:01.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa663fff640 1 --2- 172.21.15.40:0/3450799955 >> v2:172.21.15.62:3300/0 conn(0x7fa664155140 0x7fa6640cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa668d38640 1 --2- 172.21.15.40:0/3450799955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 0x7fa6640d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa6637fe640 1 -- 172.21.15.40:0/3450799955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 msgr2=0x7fa6640d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:01.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa6637fe640 1 --2- 172.21.15.40:0/3450799955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 0x7fa6640d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa6637fe640 1 -- 172.21.15.40:0/3450799955 >> v2:172.21.15.62:3300/0 conn(0x7fa664155140 msgr2=0x7fa6640cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:01.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa6637fe640 1 --2- 172.21.15.40:0/3450799955 >> v2:172.21.15.62:3300/0 conn(0x7fa664155140 0x7fa6640cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.921+0000 7fa6637fe640 1 -- 172.21.15.40:0/3450799955 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66415e6b0 con 0x7fa6640c5db0 2024-04-25T11:40:01.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.922+0000 7fa6637fe640 1 --2- 172.21.15.40:0/3450799955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6640c5db0 0x7fa66415b9a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa66406cda0 tx=0x7fa65402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77707e68173584d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.922+0000 7fa662ffd640 1 -- 172.21.15.40:0/3450799955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654037070 con 0x7fa6640c5db0 2024-04-25T11:40:01.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.922+0000 7fa662ffd640 1 -- 172.21.15.40:0/3450799955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa654032440 con 0x7fa6640c5db0 2024-04-25T11:40:01.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.922+0000 7fa662ffd640 1 -- 172.21.15.40:0/3450799955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654032830 con 0x7fa6640c5db0 2024-04-25T11:40:01.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.923+0000 7fa66a79b640 1 -- 172.21.15.40:0/3450799955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6640c5db0 msgr2=0x7fa66415b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.923+0000 7fa66a79b640 1 --2- 172.21.15.40:0/3450799955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6640c5db0 0x7fa66415b9a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa66406cda0 tx=0x7fa65402a270 comp rx=0 tx=0).stop 2024-04-25T11:40:01.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.923+0000 7fa66a79b640 1 -- 172.21.15.40:0/3450799955 shutdown_connections 2024-04-25T11:40:01.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.923+0000 7fa66a79b640 1 --2- 172.21.15.40:0/3450799955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6640c5db0 0x7fa66415b9a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.923+0000 7fa66a79b640 1 --2- 172.21.15.40:0/3450799955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 0x7fa6640d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.923+0000 7fa66a79b640 1 --2- 172.21.15.40:0/3450799955 >> v2:172.21.15.62:3300/0 conn(0x7fa664155140 0x7fa6640cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.923+0000 7fa66a79b640 1 -- 172.21.15.40:0/3450799955 >> 172.21.15.40:0/3450799955 conn(0x7fa66406c7a0 msgr2=0x7fa66406d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:01.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.923+0000 7fa66a79b640 1 -- 172.21.15.40:0/3450799955 shutdown_connections 2024-04-25T11:40:01.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.923+0000 7fa66a79b640 1 -- 172.21.15.40:0/3450799955 wait complete. 2024-04-25T11:40:01.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.924+0000 7fa66a79b640 1 Processor -- start 2024-04-25T11:40:01.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.924+0000 7fa66a79b640 1 -- start start 2024-04-25T11:40:01.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa66a79b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6640c5db0 0x7fa66414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa66a79b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa664155140 0x7fa66414be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa66a79b640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 0x7fa6640c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa66a79b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa664155520 con 0x7fa664155140 2024-04-25T11:40:01.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa66a79b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa66415f320 con 0x7fa6640c5db0 2024-04-25T11:40:01.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa66a79b640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa66414d590 con 0x7fa66415eeb0 2024-04-25T11:40:01.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa664155140 0x7fa66414be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa664155140 0x7fa66414be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46912/0 (socket says 172.21.15.40:46912) 2024-04-25T11:40:01.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa6637fe640 1 -- 172.21.15.40:0/1061504023 learned_addr learned my addr 172.21.15.40:0/1061504023 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:40:01.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa663fff640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6640c5db0 0x7fa66414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.925+0000 7fa668d38640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 0x7fa6640c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa6637fe640 1 -- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 msgr2=0x7fa6640c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa6637fe640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 0x7fa6640c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa6637fe640 1 -- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6640c5db0 msgr2=0x7fa66414df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa6637fe640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6640c5db0 0x7fa66414df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa6637fe640 1 -- 172.21.15.40:0/1061504023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6641f0de0 con 0x7fa664155140 2024-04-25T11:40:01.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa6637fe640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa664155140 0x7fa66414be60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa65402a2b0 tx=0x7fa6540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa660ff9640 1 -- 172.21.15.40:0/1061504023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654041020 con 0x7fa664155140 2024-04-25T11:40:01.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa660ff9640 1 -- 172.21.15.40:0/1061504023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa65403d620 con 0x7fa664155140 2024-04-25T11:40:01.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6641f10a0 con 0x7fa664155140 2024-04-25T11:40:01.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa660ff9640 1 -- 172.21.15.40:0/1061504023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654037040 con 0x7fa664155140 2024-04-25T11:40:01.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.926+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6641f1660 con 0x7fa664155140 2024-04-25T11:40:01.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.927+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66415be90 con 0x7fa664155140 2024-04-25T11:40:01.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.927+0000 7fa660ff9640 1 -- 172.21.15.40:0/1061504023 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa654044020 con 0x7fa664155140 2024-04-25T11:40:01.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.928+0000 7fa660ff9640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa65006c920 0x7fa65006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.928+0000 7fa660ff9640 1 -- 172.21.15.40:0/1061504023 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6255+0+0 (secure 0 0 0) 0x7fa6540ac410 con 0x7fa664155140 2024-04-25T11:40:01.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.928+0000 7fa663fff640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa65006c920 0x7fa65006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.929+0000 7fa663fff640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa65006c920 0x7fa65006ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa658009ac0 tx=0x7fa658009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:01.932+0000 7fa660ff9640 1 -- 172.21.15.40:0/1061504023 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa66415be90 con 0x7fa664155140 2024-04-25T11:40:02.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.119+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa664068bb0 con 0x7fa664155140 2024-04-25T11:40:02.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.750+0000 7fa660ff9640 1 -- 172.21.15.40:0/1061504023 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7fa65404b030 con 0x7fa664155140 2024-04-25T11:40:02.737 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:40:02.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.754+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa65006c920 msgr2=0x7fa65006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.754+0000 7fa66a79b640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa65006c920 0x7fa65006ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa658009ac0 tx=0x7fa658009190 comp rx=0 tx=0).stop 2024-04-25T11:40:02.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.754+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa664155140 msgr2=0x7fa66414be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.754+0000 7fa66a79b640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa664155140 0x7fa66414be60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa65402a2b0 tx=0x7fa6540046c0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.755+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 shutdown_connections 2024-04-25T11:40:02.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.755+0000 7fa66a79b640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fa65006c920 0x7fa65006ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.755+0000 7fa66a79b640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa66415eeb0 0x7fa6640c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.755+0000 7fa66a79b640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa664155140 0x7fa66414be60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.755+0000 7fa66a79b640 1 --2- 172.21.15.40:0/1061504023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6640c5db0 0x7fa66414df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.755+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 >> 172.21.15.40:0/1061504023 conn(0x7fa66406c7a0 msgr2=0x7fa6640d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:02.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.756+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 shutdown_connections 2024-04-25T11:40:02.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.756+0000 7fa66a79b640 1 -- 172.21.15.40:0/1061504023 wait complete. 2024-04-25T11:40:02.759 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:40:02.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.882+0000 7fc195f08640 1 Processor -- start 2024-04-25T11:40:02.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.882+0000 7fc195f08640 1 -- start start 2024-04-25T11:40:02.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.883+0000 7fc195f08640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc190152f90 0x7fc1900ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.883+0000 7fc195f08640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cae00 0x7fc190071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.883+0000 7fc195f08640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1900cb7e0 con 0x7fc1900cb240 2024-04-25T11:40:02.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.883+0000 7fc195f08640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fc19015c140 con 0x7fc190152f90 2024-04-25T11:40:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.883+0000 7fc195f08640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc19015c420 con 0x7fc1900cae00 2024-04-25T11:40:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18effd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cae00 0x7fc190071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18f7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc190152f90 0x7fc1900ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18effd640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cae00 0x7fc190071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.40:53440/0 (socket says 172.21.15.40:53440) 2024-04-25T11:40:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18ffff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc1900cb240 0x7fc190158a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.40:34046/0 (socket says 172.21.15.40:34046) 2024-04-25T11:40:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18effd640 1 -- 172.21.15.40:0/1376130158 learned_addr learned my addr 172.21.15.40:0/1376130158 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:40:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18effd640 1 -- 172.21.15.40:0/1376130158 >> v2:172.21.15.62:3300/0 conn(0x7fc190152f90 msgr2=0x7fc1900ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:02.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18effd640 1 --2- 172.21.15.40:0/1376130158 >> v2:172.21.15.62:3300/0 conn(0x7fc190152f90 0x7fc1900ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18effd640 1 -- 172.21.15.40:0/1376130158 >> v1:172.21.15.62:6789/0 conn(0x7fc1900cb240 legacy=0x7fc190158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18effd640 1 -- 172.21.15.40:0/1376130158 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19015c700 con 0x7fc1900cae00 2024-04-25T11:40:02.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.884+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/1376130158 >> v2:172.21.15.62:3300/0 conn(0x7fc190152f90 0x7fc1900ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:40:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc18effd640 1 --2- 172.21.15.40:0/1376130158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cae00 0x7fc190071460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc184009870 tx=0x7fc18402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a3c1df8f904b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc18e7fc640 1 -- 172.21.15.40:0/1376130158 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184037070 con 0x7fc1900cae00 2024-04-25T11:40:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc18e7fc640 1 -- 172.21.15.40:0/1376130158 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc18402ae50 con 0x7fc1900cae00 2024-04-25T11:40:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc18e7fc640 1 -- 172.21.15.40:0/1376130158 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184032880 con 0x7fc1900cae00 2024-04-25T11:40:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc195f08640 1 -- 172.21.15.40:0/1376130158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cae00 msgr2=0x7fc190071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc195f08640 1 --2- 172.21.15.40:0/1376130158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cae00 0x7fc190071460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc184009870 tx=0x7fc18402a300 comp rx=0 tx=0).stop 2024-04-25T11:40:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc195f08640 1 -- 172.21.15.40:0/1376130158 shutdown_connections 2024-04-25T11:40:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc195f08640 1 --2- 172.21.15.40:0/1376130158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cae00 0x7fc190071460 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc195f08640 1 --2- 172.21.15.40:0/1376130158 >> v2:172.21.15.62:3300/0 conn(0x7fc190152f90 0x7fc1900ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc195f08640 1 -- 172.21.15.40:0/1376130158 >> 172.21.15.40:0/1376130158 conn(0x7fc190070ae0 msgr2=0x7fc19006fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.885+0000 7fc195f08640 1 -- 172.21.15.40:0/1376130158 shutdown_connections 2024-04-25T11:40:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.886+0000 7fc195f08640 1 -- 172.21.15.40:0/1376130158 wait complete. 2024-04-25T11:40:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.887+0000 7fc195f08640 1 Processor -- start 2024-04-25T11:40:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.887+0000 7fc195f08640 1 -- start start 2024-04-25T11:40:02.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.887+0000 7fc195f08640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1900cae00 0x7fc1900c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc195f08640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1900cb240 0x7fc1901e4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc195f08640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cc8a0 0x7fc1901e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc195f08640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc190159120 con 0x7fc1900cb240 2024-04-25T11:40:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc195f08640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc1900c54d0 con 0x7fc1900cae00 2024-04-25T11:40:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc195f08640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc1901592a0 con 0x7fc1900cc8a0 2024-04-25T11:40:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1900cb240 0x7fc1901e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1900cb240 0x7fc1901e4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48194/0 (socket says 172.21.15.40:48194) 2024-04-25T11:40:02.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18effd640 1 -- 172.21.15.40:0/1802635299 learned_addr learned my addr 172.21.15.40:0/1802635299 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T11:40:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1900cae00 0x7fc1900c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18ffff640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cc8a0 0x7fc1901e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18effd640 1 -- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cc8a0 msgr2=0x7fc1901e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18effd640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cc8a0 0x7fc1901e46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18effd640 1 -- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1900cae00 msgr2=0x7fc1900c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18effd640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1900cae00 0x7fc1900c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18effd640 1 -- 172.21.15.40:0/1802635299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc190152410 con 0x7fc1900cb240 2024-04-25T11:40:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc18effd640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1900cb240 0x7fc1901e4160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc18402a340 tx=0x7fc1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc177fff640 1 -- 172.21.15.40:0/1802635299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184041020 con 0x7fc1900cb240 2024-04-25T11:40:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1901ef560 con 0x7fc1900cb240 2024-04-25T11:40:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc177fff640 1 -- 172.21.15.40:0/1802635299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc18403c870 con 0x7fc1900cb240 2024-04-25T11:40:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.888+0000 7fc177fff640 1 -- 172.21.15.40:0/1802635299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184037040 con 0x7fc1900cb240 2024-04-25T11:40:02.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.889+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1900cc4d0 con 0x7fc1900cb240 2024-04-25T11:40:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.890+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc190069200 con 0x7fc1900cb240 2024-04-25T11:40:02.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.890+0000 7fc177fff640 1 -- 172.21.15.40:0/1802635299 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc184044020 con 0x7fc1900cb240 2024-04-25T11:40:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.890+0000 7fc177fff640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc17c06c920 0x7fc17c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.891+0000 7fc177fff640 1 -- 172.21.15.40:0/1802635299 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5817+0+0 (secure 0 0 0) 0x7fc1840ac3e0 con 0x7fc1900cb240 2024-04-25T11:40:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.891+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc17c06c920 0x7fc17c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.891+0000 7fc18f7fe640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc17c06c920 0x7fc17c06ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc180009560 tx=0x7fc180009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:02.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:02.895+0000 7fc177fff640 1 -- 172.21.15.40:0/1802635299 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc18404b030 con 0x7fc1900cb240 2024-04-25T11:40:03.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.079+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1900cbbf0 con 0x7fc1900cb240 2024-04-25T11:40:03.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.755+0000 7fc177fff640 1 -- 172.21.15.40:0/1802635299 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7fc18407a040 con 0x7fc1900cb240 2024-04-25T11:40:03.742 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-25T11:40:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.758+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc17c06c920 msgr2=0x7fc17c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.758+0000 7fc195f08640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc17c06c920 0x7fc17c06ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc180009560 tx=0x7fc180009190 comp rx=0 tx=0).stop 2024-04-25T11:40:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1900cb240 msgr2=0x7fc1901e4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1900cb240 0x7fc1901e4160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc18402a340 tx=0x7fc1840046c0 comp rx=0 tx=0).stop 2024-04-25T11:40:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 shutdown_connections 2024-04-25T11:40:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:6832/1406383768,v1:172.21.15.62:6833/1406383768] conn(0x7fc17c06c920 0x7fc17c06ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1900cc8a0 0x7fc1901e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1900cb240 0x7fc1901e4160 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:03.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 --2- 172.21.15.40:0/1802635299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1900cae00 0x7fc1900c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:03.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 >> 172.21.15.40:0/1802635299 conn(0x7fc190070ae0 msgr2=0x7fc190151230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:03.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 shutdown_connections 2024-04-25T11:40:03.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T11:40:03.759+0000 7fc195f08640 1 -- 172.21.15.40:0/1802635299 wait complete. 2024-04-25T11:40:03.762 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T11:40:03.763 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:40:03.763 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:03.773 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:40:03.773 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:40:03.773 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:03.824 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:40:03.824 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-25T11:40:03.824 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:03.876 INFO:tasks.ceph.mds.e:Stopped 2024-04-25T11:40:03.876 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:40:03.876 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:03.887 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:40:03.888 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:40:03.888 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:03.939 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:40:03.940 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-25T11:40:03.940 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:03.991 INFO:tasks.ceph.mds.f:Stopped 2024-04-25T11:40:03.991 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T11:40:03.992 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T11:40:03.992 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.023 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T11:40:04.024 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T11:40:04.024 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.107 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T11:40:04.107 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T11:40:04.107 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.147 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T11:40:04.147 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T11:40:04.147 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.224 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T11:40:04.224 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T11:40:04.225 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.259 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T11:40:04.259 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T11:40:04.259 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.330 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T11:40:04.330 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T11:40:04.330 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.407 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T11:40:04.408 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T11:40:04.408 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.478 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T11:40:04.478 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T11:40:04.479 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T11:40:04.479 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.506 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T11:40:04.507 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T11:40:04.507 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.549 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T11:40:04.549 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T11:40:04.549 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T11:40:04.549 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.562 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T11:40:04.563 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T11:40:04.563 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.602 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T11:40:04.602 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T11:40:04.602 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:40:04.655 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T11:40:04.655 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T11:40:04.656 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-04-25T11:40:04.701 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:40:04.701 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T11:40:04.931 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:40:04.932 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T11:40:05.000 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:40:05.001 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T11:40:05.078 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:40:05.078 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T11:40:05.130 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:40:05.131 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T11:40:05.340 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:40:05.340 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T11:40:05.462 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:40:05.462 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T11:40:05.511 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:40:05.511 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T11:40:05.561 INFO:tasks.ceph:Archiving mon data... 2024-04-25T11:40:05.563 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/data/mon.a.tgz 2024-04-25T11:40:05.563 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-04-25T11:40:05.582 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.3l4SixjNJL 2024-04-25T11:40:05.582 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3l4SixjNJL 2024-04-25T11:40:05.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.3l4SixjNJL 2024-04-25T11:40:05.807 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.3l4SixjNJL is 271KB 2024-04-25T11:40:05.902 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.3l4SixjNJL 2024-04-25T11:40:05.919 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/data/mon.b.tgz 2024-04-25T11:40:05.919 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-04-25T11:40:05.935 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.Xa4OUpONn6 2024-04-25T11:40:05.935 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Xa4OUpONn6 2024-04-25T11:40:06.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.Xa4OUpONn6 2024-04-25T11:40:06.166 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.Xa4OUpONn6 is 367KB 2024-04-25T11:40:06.262 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.Xa4OUpONn6 2024-04-25T11:40:06.277 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/data/mon.c.tgz 2024-04-25T11:40:06.277 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-04-25T11:40:06.332 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.aqA9uy3l2C 2024-04-25T11:40:06.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.aqA9uy3l2C 2024-04-25T11:40:06.462 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.aqA9uy3l2C 2024-04-25T11:40:06.544 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.aqA9uy3l2C is 367KB 2024-04-25T11:40:06.641 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.aqA9uy3l2C 2024-04-25T11:40:06.656 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T11:40:06.656 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:40:06.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:40:06.744 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:40:06.775 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:40:06.775 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T11:40:06.777 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/remote/smithi040/crash 2024-04-25T11:40:06.777 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:40:06.813 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/remote/smithi062/crash 2024-04-25T11:40:06.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:40:06.838 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T11:40:06.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:40:06.858 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:40:06.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:40:06.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:40:06.986 INFO:tasks.ceph:Compressing logs... 2024-04-25T11:40:06.987 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:40:07.005 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:40:07.072 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34469.log 2024-04-25T11:40:07.072 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:40:07.072 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34760.log 2024-04-25T11:40:07.072 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.34469.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-25T11:40:07.073 INFO:teuthology.orchestra.run.smithi062.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34469.log.gz 2024-04-25T11:40:07.073 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.34760.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:40:07.073 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mds.f.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34760.log.gz 2024-04-25T11:40:07.073 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34509.log 2024-04-25T11:40:07.073 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T11:40:07.073 INFO:teuthology.orchestra.run.smithi062.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:40:07.073 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.34509.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T11:40:07.073 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34509.log.gz 2024-04-25T11:40:07.074 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T11:40:07.074 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34720.log 2024-04-25T11:40:07.074 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.5.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:40:07.074 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T11:40:07.074 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.34720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34640.log 2024-04-25T11:40:07.074 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose 86.9% -- /var/log/ceph/ceph-client.admin.34600.log -- replaced with /var/log/ceph/ceph-client.admin.34720.log.gz 2024-04-25T11:40:07.074 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T11:40:07.074 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.34640.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T11:40:07.074 INFO:teuthology.orchestra.run.smithi062.stderr:gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34640.log.gz 2024-04-25T11:40:07.075 INFO:teuthology.orchestra.run.smithi062.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T11:40:07.075 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.34600.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T11:40:07.075 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mon.c.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34600.log.gz 2024-04-25T11:40:07.075 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-25T11:40:07.075 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34680.log 2024-04-25T11:40:07.075 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34549.log 2024-04-25T11:40:07.075 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.34680.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34680.log.gz 2024-04-25T11:40:07.075 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T11:40:07.076 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.34549.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34549.log.gz 2024-04-25T11:40:07.076 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T11:40:07.078 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T11:40:07.085 INFO:teuthology.orchestra.run.smithi062.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T11:40:07.092 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mon.b.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T11:40:07.099 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41101.log 2024-04-25T11:40:07.099 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43052.log 2024-04-25T11:40:07.099 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43980.log 2024-04-25T11:40:07.099 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38748.log 2024-04-25T11:40:07.099 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41101.log.gz 2024-04-25T11:40:07.100 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43052.log.gz 2024-04-25T11:40:07.100 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39698.log 2024-04-25T11:40:07.100 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43980.log.gz 2024-04-25T11:40:07.100 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40041.log 2024-04-25T11:40:07.100 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39698.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38748.log.gz 2024-04-25T11:40:07.100 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41311.log 2024-04-25T11:40:07.100 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39698.log.gz 2024-04-25T11:40:07.100 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-04-25T11:40:07.100 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44064.log 2024-04-25T11:40:07.101 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40041.log.gz 2024-04-25T11:40:07.101 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40378.log 2024-04-25T11:40:07.101 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41311.log.gz 2024-04-25T11:40:07.101 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-04-25T11:40:07.101 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38874.log 2024-04-25T11:40:07.101 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40378.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44064.log.gz 2024-04-25T11:40:07.101 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39316.log 2024-04-25T11:40:07.101 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38874.log: gzip 88.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.40378.log.gz -- 2024-04-25T11:40:07.101 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.40260.log 2024-04-25T11:40:07.102 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39316.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38874.log.gz 2024-04-25T11:40:07.102 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41017.log 2024-04-25T11:40:07.102 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39316.log.gz 2024-04-25T11:40:07.102 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38706.log 2024-04-25T11:40:07.102 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41017.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40260.log.gz 2024-04-25T11:40:07.102 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39705.log 2024-04-25T11:40:07.102 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41017.log.gz 2024-04-25T11:40:07.102 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38790.log 2024-04-25T11:40:07.103 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41185.log 2024-04-25T11:40:07.103 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38706.log.gz 2024-04-25T11:40:07.103 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39705.log.gz 2024-04-25T11:40:07.103 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42828.log 2024-04-25T11:40:07.103 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38790.log.gz 2024-04-25T11:40:07.103 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41185.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:40:07.103 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42828.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41185.log.gz 2024-04-25T11:40:07.103 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43010.log 2024-04-25T11:40:07.103 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42828.log.gz 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40380.log 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39701.log 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41824.log 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43010.log.gz 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr: 84.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40380.log.gz 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41656.log 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41824.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39701.log.gz 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38962.log 2024-04-25T11:40:07.104 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41824.log.gz 2024-04-25T11:40:07.105 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.41656.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43526.log 2024-04-25T11:40:07.105 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38832.log 2024-04-25T11:40:07.105 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43526.log: gzip -5 --verbose 87.8% -- -- replaced with /var/log/ceph/ceph-client.admin.38962.log.gz /var/log/ceph/ceph-client.admin.34784.log 2024-04-25T11:40:07.105 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-25T11:40:07.105 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41656.log.gz 2024-04-25T11:40:07.105 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43526.log.gz 2024-04-25T11:40:07.105 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38832.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T11:40:07.105 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42246.log 2024-04-25T11:40:07.105 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38832.log.gz 2024-04-25T11:40:07.106 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34784.log.gz 2024-04-25T11:40:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43358.log 2024-04-25T11:40:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.42246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38522.log 2024-04-25T11:40:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43610.log 2024-04-25T11:40:07.106 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43358.log.gz 2024-04-25T11:40:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43736.log 2024-04-25T11:40:07.106 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38522.log.gz 2024-04-25T11:40:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43610.log: /var/log/ceph/ceph-client.admin.39274.log 2024-04-25T11:40:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43736.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:40:07.107 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.0.42246.log.gz 2024-04-25T11:40:07.107 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43610.log.gz 2024-04-25T11:40:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39274.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43736.log.gz 2024-04-25T11:40:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39442.log 2024-04-25T11:40:07.107 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39274.log.gz 2024-04-25T11:40:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44271.log 2024-04-25T11:40:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41269.log 2024-04-25T11:40:07.107 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39442.log.gz 2024-04-25T11:40:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41440.log 2024-04-25T11:40:07.108 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:40:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43095.log 2024-04-25T11:40:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41269.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44271.log.gz 2024-04-25T11:40:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42518.log 2024-04-25T11:40:07.108 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41269.log.gz 2024-04-25T11:40:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43232.log 2024-04-25T11:40:07.108 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41440.log.gz 2024-04-25T11:40:07.108 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43095.log.gz 2024-04-25T11:40:07.109 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34417.log 2024-04-25T11:40:07.109 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40618.log 2024-04-25T11:40:07.109 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43232.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42518.log.gz 2024-04-25T11:40:07.109 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38405.log 2024-04-25T11:40:07.109 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43232.log.gz 2024-04-25T11:40:07.109 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40618.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T11:40:07.109 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.34417.log.gz 2024-04-25T11:40:07.109 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40618.log.gz 2024-04-25T11:40:07.109 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41227.log 2024-04-25T11:40:07.110 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38405.log.gz 2024-04-25T11:40:07.110 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40377.log 2024-04-25T11:40:07.110 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41398.log 2024-04-25T11:40:07.110 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40377.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41227.log.gz 2024-04-25T11:40:07.110 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39707.log 2024-04-25T11:40:07.110 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40377.log.gz 2024-04-25T11:40:07.110 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40035.log 2024-04-25T11:40:07.110 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34681.log 2024-04-25T11:40:07.111 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41398.log.gz 2024-04-25T11:40:07.111 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40660.log 2024-04-25T11:40:07.111 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39707.log.gz 2024-04-25T11:40:07.111 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40035.log.gz 2024-04-25T11:40:07.111 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41143.log 2024-04-25T11:40:07.111 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34681.log.gz 2024-04-25T11:40:07.111 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T11:40:07.111 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40660.log: /var/log/ceph/ceph-client.admin.41143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42702.log 2024-04-25T11:40:07.111 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40660.log.gz 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.c.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41143.log.gz 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42786.log 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40803.log 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42702.log: 91.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43820.log 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42702.log.gz 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42786.log.gz 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43694.log 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40803.log: gzip/var/log/ceph/ceph-client.admin.43820.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39695.log 2024-04-25T11:40:07.112 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40803.log.gz 2024-04-25T11:40:07.113 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43694.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43820.log.gz 2024-04-25T11:40:07.113 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39190.log 2024-04-25T11:40:07.113 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.43694.log.gz -5 2024-04-25T11:40:07.113 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T11:40:07.113 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39695.log: /var/log/ceph/ceph-client.admin.39190.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39695.log.gz 2024-04-25T11:40:07.113 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T11:40:07.113 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39190.log.gz 2024-04-25T11:40:07.113 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40043.log 2024-04-25T11:40:07.114 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44022.log 2024-04-25T11:40:07.114 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39703.log 2024-04-25T11:40:07.114 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40040.log 2024-04-25T11:40:07.114 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40043.log.gz 2024-04-25T11:40:07.114 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38622.log 2024-04-25T11:40:07.114 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44022.log.gz 2024-04-25T11:40:07.114 INFO:teuthology.orchestra.run.smithi040.stderr:gzip 88.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.39046.log 2024-04-25T11:40:07.114 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39703.log.gz/var/log/ceph/ceph-client.admin.40040.log: 2024-04-25T11:40:07.114 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38622.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T11:40:07.115 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40040.log.gz 2024-04-25T11:40:07.115 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.38622.log.gz 2024-04-25T11:40:07.115 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39526.log 2024-04-25T11:40:07.115 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39046.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41059.log 2024-04-25T11:40:07.115 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39046.log.gz 2024-04-25T11:40:07.115 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41590.log 2024-04-25T11:40:07.115 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39526.log.gz 2024-04-25T11:40:07.115 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39130.log 2024-04-25T11:40:07.115 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% 92.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T11:40:07.116 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41059.log.gz 2024-04-25T11:40:07.116 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40037.log 2024-04-25T11:40:07.116 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41590.log: /var/log/ceph/ceph-client.admin.39130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38918.log 2024-04-25T11:40:07.116 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%/var/log/ceph/ceph-client.admin.40037.log: -- replaced with /var/log/ceph/ceph-client.admin.39130.log.gz 2024-04-25T11:40:07.116 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44107.log 2024-04-25T11:40:07.116 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41590.log.gz 2024-04-25T11:40:07.116 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40037.log.gz 2024-04-25T11:40:07.116 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38918.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38918.log.gz 2024-04-25T11:40:07.116 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42744.log 2024-04-25T11:40:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43568.log 2024-04-25T11:40:07.117 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44107.log.gz 2024-04-25T11:40:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41353.log 2024-04-25T11:40:07.117 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42744.log.gz 2024-04-25T11:40:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39400.log 2024-04-25T11:40:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41353.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43568.log.gz 2024-04-25T11:40:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43442.log 2024-04-25T11:40:07.117 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41353.log.gz 2024-04-25T11:40:07.117 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T11:40:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39400.log: /var/log/ceph/ceph-client.admin.43442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40975.log 2024-04-25T11:40:07.118 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39400.log.gz 2024-04-25T11:40:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mon.a.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43442.log.gz 2024-04-25T11:40:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40891.log 2024-04-25T11:40:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44149.log 2024-04-25T11:40:07.118 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3%/var/log/ceph/ceph-client.admin.40891.log: -- replaced with /var/log/ceph/ceph-client.admin.40975.log.gz 2024-04-25T11:40:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40761.log 2024-04-25T11:40:07.118 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40891.log.gz 2024-04-25T11:40:07.118 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43652.log 2024-04-25T11:40:07.119 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43940.log 2024-04-25T11:40:07.119 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44149.log.gz 2024-04-25T11:40:07.119 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40761.log: /var/log/ceph/ceph-client.admin.43652.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28345.log 2024-04-25T11:40:07.119 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40761.log.gz 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43652.log.gz 2024-04-25T11:40:07.119 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-25T11:40:07.119 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43940.log: gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43940.log.gz 2024-04-25T11:40:07.119 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.42660.log 2024-04-25T11:40:07.119 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.tmp-client.admin.28345.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28345.log.gz 2024-04-25T11:40:07.119 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43880.log 2024-04-25T11:40:07.120 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42660.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42660.log.gz 2024-04-25T11:40:07.120 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43400.log 2024-04-25T11:40:07.120 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44313.log 2024-04-25T11:40:07.120 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43880.log.gz 2024-04-25T11:40:07.120 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43400.log: gzip -5 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43400.log.gz 2024-04-25T11:40:07.120 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41524.log 2024-04-25T11:40:07.120 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39706.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44313.log.gz 2024-04-25T11:40:07.120 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-25T11:40:07.120 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44227.log 2024-04-25T11:40:07.121 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40375.log 2024-04-25T11:40:07.121 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41524.log.gz 2024-04-25T11:40:07.121 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39706.log.gz 2024-04-25T11:40:07.121 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43316.log 2024-04-25T11:40:07.121 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44227.log: /var/log/ceph/ceph-client.admin.40375.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44227.log.gz 2024-04-25T11:40:07.121 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40375.log.gz 2024-04-25T11:40:07.121 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39484.log 2024-04-25T11:40:07.121 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40933.log 2024-04-25T11:40:07.121 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39484.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43316.log.gz 2024-04-25T11:40:07.122 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-25T11:40:07.122 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39484.log.gz 2024-04-25T11:40:07.122 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40933.log: gzip -5 87.5% --verbose -- /var/log/ceph/ceph-client.admin.40039.log 2024-04-25T11:40:07.122 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40933.log.gz 2024-04-25T11:40:07.122 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40710.log 2024-04-25T11:40:07.122 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42434.log 2024-04-25T11:40:07.122 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40039.log.gz 2024-04-25T11:40:07.122 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39004.log 2024-04-25T11:40:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42434.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40710.log.gz 2024-04-25T11:40:07.123 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42434.log.gz 2024-04-25T11:40:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42560.log 2024-04-25T11:40:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39004.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39004.log.gz 2024-04-25T11:40:07.123 INFO:teuthology.orchestra.run.smithi040.stderr: 90.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40044.log 2024-04-25T11:40:07.123 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-25T11:40:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39358.log 2024-04-25T11:40:07.123 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42476.log 2024-04-25T11:40:07.123 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42560.log.gz 2024-04-25T11:40:07.124 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40044.log: /var/log/ceph/ceph-client.admin.39358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40376.log 2024-04-25T11:40:07.124 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40044.log.gz 2024-04-25T11:40:07.124 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39358.log.gz/var/log/ceph/ceph-client.admin.42476.log: 2024-04-25T11:40:07.124 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42968.log 2024-04-25T11:40:07.124 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42476.log.gz 2024-04-25T11:40:07.124 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40259.log 2024-04-25T11:40:07.124 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40376.log: /var/log/ceph/ceph-client.admin.42968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39232.log 2024-04-25T11:40:07.124 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40259.log: 89.2% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40376.log.gz 2024-04-25T11:40:07.124 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42968.log.gz 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40259.log.gz 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43274.log 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39232.log.gz 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40849.log 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43274.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43274.log.gz 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39708.log 2024-04-25T11:40:07.125 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43778.log 2024-04-25T11:40:07.126 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40849.log: /var/log/ceph/ceph-client.admin.39708.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40849.log.gz 2024-04-25T11:40:07.126 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39708.log.gzgzip 2024-04-25T11:40:07.126 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39088.log 2024-04-25T11:40:07.126 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40379.log 2024-04-25T11:40:07.126 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.39088.log: /var/log/ceph/ceph-client.admin.43778.log: 89.3% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43778.log.gz 2024-04-25T11:40:07.126 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39088.log.gz 2024-04-25T11:40:07.126 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40042.log 2024-04-25T11:40:07.126 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41722.log 2024-04-25T11:40:07.127 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40379.log.gz 2024-04-25T11:40:07.127 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40576.log 2024-04-25T11:40:07.127 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.41722.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40042.log.gz 2024-04-25T11:40:07.127 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42602.log 2024-04-25T11:40:07.127 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.40576.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41722.log.gz 2024-04-25T11:40:07.127 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40576.log.gz 2024-04-25T11:40:07.127 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43484.log 2024-04-25T11:40:07.127 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.42602.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42602.log.gz 2024-04-25T11:40:07.127 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43484.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43484.log.gz 2024-04-25T11:40:07.128 INFO:teuthology.orchestra.run.smithi040.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T11:40:07.134 INFO:teuthology.orchestra.run.smithi062.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T11:40:07.183 INFO:teuthology.orchestra.run.smithi062.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T11:40:07.246 INFO:teuthology.orchestra.run.smithi040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T11:40:07.585 INFO:teuthology.orchestra.run.smithi040.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T11:40:07.590 INFO:teuthology.orchestra.run.smithi062.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T11:40:07.595 INFO:teuthology.orchestra.run.smithi062.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T11:40:07.602 INFO:teuthology.orchestra.run.smithi040.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T11:40:07.603 INFO:teuthology.orchestra.run.smithi040.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T11:40:07.608 INFO:teuthology.orchestra.run.smithi040.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T11:40:07.611 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-25T11:40:07.611 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m0.592s 2024-04-25T11:40:07.611 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m2.424s 2024-04-25T11:40:07.611 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m0.173s 2024-04-25T11:40:07.629 INFO:teuthology.orchestra.run.smithi062.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T11:40:07.648 INFO:teuthology.orchestra.run.smithi062.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T11:40:07.650 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T11:40:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:real 0m0.609s 2024-04-25T11:40:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:user 0m2.449s 2024-04-25T11:40:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m0.092s 2024-04-25T11:40:07.651 INFO:tasks.ceph:Archiving logs... 2024-04-25T11:40:07.651 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/remote/smithi040/log 2024-04-25T11:40:07.652 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:40:08.114 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/remote/smithi062/log 2024-04-25T11:40:08.115 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:40:08.494 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T11:40:08.508 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T11:40:08.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:40:08.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:40:08.591 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T11:40:08.600 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T11:40:08.601 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:40:08.603 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:40:08.616 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:40:08.618 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:40:08.621 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:40:08.621 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-25T11:40:08.621 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +661us[ +673us] +/- 40ms 2024-04-25T11:40:08.621 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -562us[ -562us] +/- 79ms 2024-04-25T11:40:08.621 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -694us[ -694us] +/- 78ms 2024-04-25T11:40:08.622 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:40:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:40:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-25T11:40:08.623 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +671us[ +672us] +/- 40ms 2024-04-25T11:40:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -566us[ -565us] +/- 79ms 2024-04-25T11:40:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -704us[ -703us] +/- 78ms 2024-04-25T11:40:08.624 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:40:08.625 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T11:40:08.635 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T11:40:08.635 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T11:40:08.644 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:40:08.728 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:40:08.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:40:08.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:40:08.818 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:40:08.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:40:08.846 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T11:40:08.857 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T11:40:08.865 INFO:teuthology.task.internal:Duration was 486.852637 seconds 2024-04-25T11:40:08.866 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T11:40:08.877 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T11:40:08.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:40:08.880 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:40:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:40:08.924 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:40:09.287 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T11:40:09.287 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T11:40:09.287 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:40:09.310 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T11:40:09.311 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:40:09.351 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T11:40:09.351 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:40:09.354 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:40:09.417 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T11:40:09.418 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:40:09.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:40:09.553 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T11:40:09.564 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T11:40:09.564 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:40:09.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:40:09.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T11:40:09.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:40:09.631 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:40:09.651 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-04-25T11:40:09.675 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-04-25T11:40:09.693 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:40:09.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:40:09.719 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:40:09.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:40:09.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T11:40:09.759 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T11:40:09.759 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/remote/smithi040 2024-04-25T11:40:09.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:40:09.792 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332/remote/smithi062 2024-04-25T11:40:09.792 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:40:09.824 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T11:40:09.824 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:40:09.829 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:40:09.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T11:40:09.886 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T11:40:09.887 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T11:40:09.897 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T11:40:09.897 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:40:09.901 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:40:09.912 INFO:teuthology.orchestra.run.smithi040.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:40 /home/ubuntu/cephtest 2024-04-25T11:40:09.928 INFO:teuthology.orchestra.run.smithi062.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:40 /home/ubuntu/cephtest 2024-04-25T11:40:09.929 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T11:40:09.939 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T11:40:09.962 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T11:40:09.989 DEBUG:teuthology.nuke:shortname: smithi040 2024-04-25T11:40:09.989 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:40:10.011 DEBUG:teuthology.nuke:shortname: smithi062 2024-04-25T11:40:10.011 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:40:10.014 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:26:32.830490', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJNl3N2R7LyI0uZGFSlGW1HbkwyyfKdvKA0bEJ5k3GWtbtg5eNAT09UYtGbAozRjlBR7SLesr+YXP/WxnAt3IM='} 2024-04-25T11:40:10.036 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673332', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:26:32.828375', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJktnmc6KegBeiC88dl+vdB9DEb/41t0qgszN4lGnfkUjzaK+GY6tk2mzVIgxPl2AmEqcIYAtrV7imk9k1IzJ0k='} 2024-04-25T11:40:10.066 INFO:teuthology.orchestra.console.smithi040:Power off 2024-04-25T11:40:10.066 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T11:40:10.091 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2024-04-25T11:40:10.091 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:40:10.109 INFO:teuthology.orchestra.console.smithi062:Power off 2024-04-25T11:40:10.109 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T11:40:10.118 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-04-25T11:40:10.133 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2024-04-25T11:40:10.133 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:40:10.155 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-25T11:40:14.119 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:40:14.230 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-04-25T11:40:14.248 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:40:14.358 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-25T11:40:18.232 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:40:18.343 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-04-25T11:40:18.359 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:40:18.469 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-25T11:40:22.346 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:40:22.456 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2024-04-25T11:40:22.470 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:40:22.557 INFO:teuthology.orchestra.console.smithi040:Power off completed 2024-04-25T11:40:22.581 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2024-04-25T11:40:22.682 INFO:teuthology.orchestra.console.smithi062:Power off completed 2024-04-25T11:40:22.787 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/suites/pjd}} duration: 486.8526372909546 failure_reason: '''cluster''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b453a7003ff14de5865b8c074fd41511 status: fail success: false 2024-04-25T11:40:22.787 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:40:22.869 INFO:teuthology.run:FAIL